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ABSTRACT 


Demonstration  of  required  reliability  performance  levels  prior  to  system  fielding 
has  remained  a  challenge  for  the  Army,  and  in  recent  years,  the  success  rate  of  systems 
achieving  their  stated  reliability  performance  in  operational  tests  has  declined. 
Realization  of  required  reliability  performance  necessitates  effective  management 
strategies  and  techniques  in  order  to  reduce  risks.  “Designing-in”  reliability  up  front  is 
one  of  the  critical  elements  to  insure  adequate  system  level  reliability.  It  has  also  been 
ranked  as  one  the  top  reliability  problems  by  Program  Managers.  One  design  technique 
for  maximizing  inherent  reliability  is  through  the  use  of  highly  reliable  parts,  materials, 
and  processes.  Reliable  parts,  materials,  and  processes  are  the  building  blocks  of  the  total 
system  reliability.  They  have  a  significant  role  in  overall  program  success  because  it 
contributes  to  the  ability  to  meet  its  cost,  schedule  and  performance  goals. 

This  research  concentrates  on  the  impacts  of  highly  reliable  parts  and  materials  on 
the  overall  reliability  of  a  weapon  system.  To  gather  these  data,  the  researcher  drew 
directly  from  experiences  of  part,  material,  and  process  professionals  from  the  Anny, 
Navy,  Air  Force,  and  contractors  with  whom  they  interface.  Results  show  that  the  key  to 
success  resides  in  early  involvement  of  part,  material,  and  process  experts  along  with  a 
process  that  facilitates  communication  and  open  dialogue.  Maximizing  inherent 
reliability  is  the  desired  end  state. 
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I.  INTRODUCTION 


A.  BACKGROUND  DISCUSSION 

The  Army  Vision,  announced  in  October  1999,  is  to  transition  the  entire  Army 
into  a  force  that  is  strategically  responsive  and  dominant  at  every  point  on  the  spectrum 
of  operations.  This  equates  to  a  need  for  high  readiness  levels  for  rapid  deployment,  as 
well  as  a  significantly  reduced  logistics  footprint  in  the  battlespace  without  jeopardizing 
combat  capability.  One  enabler  for  achieving  this  is  highly  reliable  systems.  Highly 
reliable  systems  are  force  effectiveness  multipliers,  as  the  resulting  benefits  contribute 
towards  reduced  maintenance  times,  increased  system  availability,  reduced  training  and 
manpower,  fewer  spare  parts,  and  a  net  reduction  in  total  ownership  costs  (TOC)  that  will 
free  up  scarce  funds  needed  for  Army  modernization. 

Demonstration  of  required  reliability  performance  levels  prior  to  system  fielding 
has  remained  a  challenge  for  the  Army.  According  to  the  Anny  Test  and  Evaluation 
Command  (ATEC),  the  success  rate  for  systems  in  operational  testing  over  a  five-year 
period  from  1996  to  2000  was  only  20  percent.  [Ref.  1]  The  failure  of  a  weapon  system 
to  meet  its  predicted  reliability  levels  could  have  a  severe  impact  on  its  future  logistics 
and  life-cycle  support  costs.  According  to  a  recent  analysis  performed  on  the  Army 
Comanche  helicopter,  missing  predicted  reliability  goals  by  just  one  percent  could  result 
in  an  increase  in  lifecycle  O&S  costs  of  over  $75  million.  [Ref.  2] 

The  reliability  development  process  is  composed  of  many  elements  (e.g.,  a  PM&P 
Program,  Materials  and  Processes,  Test  and  Evaluation,  Design,  Design  Interface, 
utilization,  etc.),  including  the  management  process  of  establishing  a  reliability  factor  for 
the  system.  This  management  process  is  such  a  crucial  part  of  the  development  process 
because,  as  reliability  is  allocated  among  the  subsystems,  it  becomes  a  design  driver.  A 
program  can  have  a  strong  PM&P  Program,  T&E,  etc.,  but  if  an  unrealistic  reliability 
factor  is  established,  the  program  will  not  have  a  reliable  system,  nor  will  it  have 
achieved  its  reliability  requirement.  As  a  rule,  reliability  is  generally  overstated,  and  the 
impacts  on  test,  support,  inventory,  personnel,  and  training  can  be  harmful.  [Ref.  3] 
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“Designing  in”  reliability  up  front  is  one  of  the  critical  elements  to  ensure 
adequate  system  level  reliability.  It  has  also  been  ranked  as  one  the  top  reliability 
problems  by  Program  Managers.  [Ref.  1]  One  design  technique  for  maximizing  inherent 
reliability  is  the  use  of  highly  reliable  parts,  materials,  and  processes.  Reliable  parts, 
materials,  and  processes  have  a  significant  role  in  overall  program  success  because  they 
contribute  to  the  ability  to  meet  cost,  schedule  and  performance  goals.  One  of  the 
techniques  to  assure  the  use  of  highly  reliable  parts,  materials,  and  processes  is  through 
the  use  of  a  Parts,  Materials,  and  Processes  (PM&P)  Program. 

A  PM&P  Program  can  be  applied  in  several  important  ways  over  the  life  cycle 
of  a  system.  An  effective  PM&P  Program  is  key  to  mitigating  risk  associated  with  the 
use  and  disposition  of  noncompliant  hardware,  avoiding  costly  redesigns,  attaining  costs 
savings  from  reduced  need  for  failure  analysis,  rework,  etc.  Throughout  this  thesis,  when 
referring  to  a  PM&P  Program,  the  capitalized  acronym  will  be  used.  When  referring  to 
the  parts,  materials,  and  processes  themselves,  no  capitalization  will  be  used  and  the 
words  will  be  spelled  out. 

B.  OBJECTIVES  AND  PURPOSE  OF  THE  RESEARCH 

Required  reliability  performance  depends  upon  effective  management  strategies 
and  techniques  to  address  reliability  risks  over  the  course  of  a  weapon  system’s 
development  and  fielding.  This  research  concentrates  on  the  impacts  of  highly  reliable 
parts  and  materials  on  the  overall  reliability  of  a  weapon  system.  In  DoD  terms,  it  will 
focus  on  essential  steps  that  a  Program  or  Logistics  Manager  can  take  to  maximize 
system  component  reliability.  A  large  part  of  the  effort  focuses  on  giving  the  reader  an 
understanding  of  how  an  acceptable  reliability  factor  is  established  for  the  system,  the 
importance  of  achieving  high  reliability,  and  the  relationship  among  reliability,  logistics 
and  life-cycle  support  costs.  The  objective  is  to  detennine  how  to  best  maximize  the 
inherent  reliabilities  of  system  components  through  the  use  of  highly  reliable  parts  and 
materials. 

C.  RESEARCH  QUESTIONS 

The  Primary  research  question  is  twofold:  how  is  the  selection  of  highly  reliable 
parts  and  materials  an  essential  part  of  the  overall  reliability  development  process,  and 
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what  critical  tactics  and  strategies  can  Program  and  Logistics  Managers  use  to  maximize 
the  inherent  reliabilities  of  system  components? 


Subsidiary  research  questions  are: 

1 .  What  technology,  tools,  and  techniques  are  available  to  ensure  the  proper 
selection  of  reliable  parts  and  materials? 

2.  How  has  acquisition  reform  changed  the  way  a  PM&P  Program  is  managed? 

3.  To  what  extent  does  commercial  industry  differ  in  its  approach  to  part  and 
material  selection,  and  can  the  Military  leverage  these  best  practices  to 
improve  performance  in  military  systems? 

4.  How  can  a  Program  and  Logistics  Manager  mitigate  part  and  material  risks? 

D.  SCOPE 

The  thesis  focuses  on  the  tactics  and  strategies  that  Program  and  Logistics 
Managers  can  use  to  ensure  the  highest  possible  inherent  reliabilities  of  system 
components.  Although  the  data  will  originate  primarily  from  Army  acquisition 
professionals,  lessons  learned  should  be  applicable  to  the  other  DoD  Services. 

The  scope  is  limited  to  an  evaluation  of  reliability  considerations  in  the  system 
design  process  from  several  aspects:  (1)  the  technology,  tools,  and  techniques  that  are 
available  to  ensure  the  proper  selection  and  use  of  highly  reliable  parts  and  materials;  (2) 
an  analysis  of  how  parts  and  materials  are  managed  under  acquisition  reform;  and  (3)  the 
extent  to  which  commercial  industry  differs  in  its  approach  towards  part  and  material 
selection  and  how  the  DoD  can  leverage  these  best  practices.  The  thesis  concludes  with  a 
recommendation  for  actions  or  steps  a  DOD  Program  and  Logistics  Manager  might  take 
to  maximize  inherent  reliability  through  the  use  of  highly  reliable  components. 

E.  METHODOLOGY 

The  methodology  used  in  this  thesis  research  consists  primarily  of  two  steps.  The 
first  step  is  to  demonstrate  why  a  weapon  system  with  a  high  inherent  reliability  must  be 
designed  using  highly  reliable  parts  and  materials.  The  importance  of  selecting  an 
acceptable  reliability  requirement  will  be  established.  The  connection  will  be  made 
between  part  and  material,  reliability,  logistics,  and  life-cycle  support  costs.  Then  the 
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thesis  explains  the  benefit  of  “designing  in”  reliability  and  how  the  selection  of  flawed 
system  level  reliability  affects  design  at  the  lower  levels.  Next,  the  thesis  provides  an 
overview  of  the  contemporary  reliability  environment  within  the  DoD.  Acquisition 
reform,  current  DoD  PM&P  Program  initiatives,  commercial  best  practices,  and 
contracting  methods  for  effective  part  and  material  selection  are  evaluated  using  literature 
reviews  and  interviews  with  acquisition  professionals.  The  thesis  also  addresses  program 
management  techniques  in  the  same  manner.  A  comprehensive  literature  review  on  the 
subject  of  reliability  encompasses  material  and  sources  that  include,  but  are  not  limited 
to: 

1.  DoD,  Army,  Navy,  and  Air  Force  publications 

2.  Published  academic  research  papers 

3.  References,  publications,  and  electronic  media  available  at  the  Naval 
Postgraduate  School  (NPS) 

4.  World  Wide  Web  Sources  (DoD,  commercial,  academic) 

5.  Personal  communication  with  School  of  Business  and  Public  Policy  faculty  at 
NPS 

The  second  step  entails  interviews  with  subject  matter  experts  (SMEs)  in  the 
parts,  materials,  and  processes  functional  area.  Data  gathering  and  analysis  were 
conducted  through  structured  interviews.  The  analysis  synthesized  feedback  and 
searched  for  collective  issues  from  the  various  SMEs  on  managing  PM&P  Program 
requirements,  the  coordination  that  is  required,  the  common  issues,  reasons  why  they 
occur,  and  how  these  risks  can  be  reduced.  To  address  the  concern  of  data  reliability,  the 
SME  had  editorial  control.  The  interview  answers  were  typed  and  given  to  the  SME  for 
review  and  approval  prior  to  incorporation. 

F.  ORGANIZATION  OF  THE  STUDY 

Chapter  I.  Introduction:  identifies  the  focus  and  purpose  of  this  thesis  as,  well  as 
the  primary  and  subsidiary  research  questions. 

Chapter  II:  Overview  and  Background:  provides  a  literature  review,  giving  the 
reader  a  basic  understanding  of  the  importance  of  establishing  an  acceptable  reliability 
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factor  for  the  system,  achieving  high  reliability,  and  the  relationship  among  reliability, 
logistics  and  life-cycle  support. 

Chapter  III:  Managing  a  PM&P  Program  on  a  Weapon  System:  presents  the  data 
collected  from  interviews  with  subject  matter  experts  (SMEs)  in  the  parts,  materials,  and 
processes  functional  area. 

Chapter  IV:  PM&P  Program  Analysis  and  Lessons  Learned:  analyzes  and 
compiles  the  PM&P  Program  technologies  and  tools,  the  impact  of  acquisition 
streamlining,  commercial  PM&P  Programs,  and  the  techniques  to  mitigate  part  and 
material  risks. 

Chapter  V:  Conclusions  and  Recommendations:  summarizes  the  findings  of  the 
research  and  answers  the  research  questions. 
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II.  BACKGROUND 


A.  INTRODUCTION 

The  purpose  of  this  chapter  is  to  provide  insight  into  how  a  weapon  system 
reliability  requirement  is  developed  and  to  explain  its  relationship  to  highly  reliable  parts, 
materials,  and  processes.  The  chapter  first  discusses  the  current  reliability  trend  in  the 
Anny.  It  then  describes  the  PM&P  Program  as  it  relates  to  the  acquisition  process  in 
order  to  make  the  analysis  presented  Chapter  IV  more  understandable.  Next,  the  chapter 
details  the  critical  need  to  detennine  the  proper  reliability  requirement  for  the  system.  It 
then  discusses  the  importance  of  achieving  high  reliability  and  the  relationships  among 
reliability,  logistics  and  life-cycle  support  costs.  The  benefits  of  “designing  in”  reliability 
and  how  those  benefits  relate  to  the  PM&P  Program  are  reviewed,  along  with  the  impact 
of  the  system-level  reliability  requirement  on  the  individual  parts,  materials,  and 
processes.  Former  and  current  DoD  policies,  as  well  as  commercial  PM&P  Program 
practices,  are  covered. 

B.  THE  CURRENT  RELIABILITY  TREND  WITHIN  THE  ARMY 

According  to  the  Army  Test  and  Evaluation  Command  (ATEC),  the  success  rate 
for  Army  systems  either  in  development  or  operational  testing  over  the  five-year  period 
from  1996  through  2000  was  only  41%.  In  the  same  five-year  period,  the  system 
operational  test  (OT)  success  rate  with  respect  to  reliability  was  only  20%.  [Ref.  4] 
Figure  1  represents  operational  test  events  that  were  used  as  the  basis  for  demonstrating 
reliability  requirements.  All  acquisition  category  (ACAT)  levels  are  represented  here. 
The  types  of  OT  events  included:  field  exercises;  IOTs,  FOTs,  LUTs;  and  combined 
DT/OT.  Points  above  the  diagonal  achieved  their  reliability  requirement  during  testing, 
while  those  below  did  not.  Most  of  the  systems  failed  to  achieve  their  reliability 
requirement  in  OT,  and  the  trend  appears  to  be  downward. 
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Figure  1 .  Army  System  Reliability  Performance  [From  Ref.  5] 

C.  DESCRIPTION  OF  A  PM&P  PROGRAM 

There  is  no  universal  definition  of  a  Parts,  Materials,  and  Processes  (PM&P) 
Program,  although  individual  definitions  for  parts,  materials,  and  processes  are  clearly 
defined  in  DOD  specifications  (e.g.,  MIL-STD-1547  and  MIL-STD-965).  For  the 
purposes  of  this  research,  a  Parts,  Materials,  and  Processes  (PM&P)  Program  is  defined 
as: 

A  process  that  deals  with  individual  Parts,  Materials,  and  Processes 
requirements.  [Ref.  6] 


In  order  to  supplement  the  earlier  definition  of  a  PM&P  Program,  a  part  is  further 
defined  as: 

One  piece,  or  two  or  more  pieces  joined  together,  which  is  not 
normally  subject  to  disassembly  without  destruction  or  impairment 
of  intended  design  use.  [Ref.  7] 
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Requirements  are  defined  as: 

Characteristics  that  individual  items  must  have  within  a  stated 
environment  or  a  set  of  conditions  as  derived  from  the  systems 
engineering  process.  [Ref.  6] 

The  conditions  or  environments  mentioned  above  are  based  on  the  ORD-derived 
constraints  and  performance  requirements,  as  reflected  onto  the  individual  part,  material, 
or  process.  Parts,  materials,  and  processes  are  critical  to  most  acquisitions  and  are  a 
fundamental  element  in  the  achievement  of  designed  hardware  perfonnance,  which 
includes: 

1 .  Analysis  to  support  the  establishment  of  comprehensive  PM&P  Program 
characteristics  from  the  derived  system  engineering  constraints  and  performance 
needs. 

2.  Establishment  of  a  cross-functional  Plan  for  efficient  and  uniform 
implementation  of  a  PM&P  Program. 

3.  Analysis  and  documentation  to  support  individual  part,  material,  and  process 
needs,  consistent  with  PM&P  Program  requirements. 

4.  Development,  maintenance,  and  control  of  databases  documenting  design 
requirements,  design  baseline,  control  and  use  of  life-limited  (items  having  a 
finite  shelf  life),  the  basis  for  decisions  made,  and  control  of  evolving 
requirements,  so  that  the  impact  of  changes  can  be  effectively  detennined  prior  to 
implementation. 

5.  Development  of  a  closed-loop  system  to  efficiently  feed  back  necessary 
changes  derived  from  system-level  performance  results  and  industry  data 
interchange. 

6.  Development  of  In-Process  or  End-of-Line  validation  processes  for  known 
good  parts,  materials,  and  processes. 
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7.  Development,  maintenance,  and  control  of  program-compliant  parts,  material, 

and  process  lists. 

A  PM&P  Program  is  applied  in  several  important  ways  over  the  life  cycle  of  a 
weapon  system.  For  example,  during  the  Concept  and  Technology  Development  and 
System  Development  and  Demonstration  phases  of  a  weapon  system,  a  PM&P  Program 
is  applied  iteratively  as  an  integral  part  of  system  engineering  to  mitigate  risk  and 
tradeoffs  between  system  performance,  ease  of  maintenance  and  manufacture,  cost, 
production  level,  parts  and  materials  availability,  and  technological  advancements. 
Ineffective  use  of  parts,  materials,  and  processes  at  this  early  stage  of  a  program  may 
cause  unnecessary  future  program  costs  and  delays.  [Ref.  6] 

D.  ESTABLISHING  AN  ACCEPTABLE  RELIABILITY  REQUIREMENT 

1.  Origination  of  the  Reliability  Requirement 

Reliability  has  been  defined  as  “the  probability  that  a  system  will  operate  within 
the  designed  specified  limits  for  a  particular  period  of  time  when  used  in  the  manner 
intended”.  [Ref.  8]  In  the  Army,  the  reliability  requirement  starts  with  the  Combat 
Developer  (CBTDEV).  The  CBTDEV  develops  the  Operational  Requirements 
Document  (ORD)  and,  hence,  is  ultimately  responsible  for  defining  the  system’s 
reliability  requirements.  Typically,  these  are  defined  in  terms  of  operational  availability 
and  mission  duration  needs. 

Reliability  requirements,  however,  are  not  developed  in  a  vacuum.  Developing 
quantitative  operational  reliability  requirements,  like  all  other  ORD  requirements,  is  a 
collaborative  process  between  the  CBTDEV  and  the  Materiel  Developer  (MATDEV) 
using  Integrated  Product  Teams  (IPTs).  IPTs  provide  a  balanced  solution  between  the 
best  estimate  of  what  will  meet  the  user’s  effectiveness,  suitability,  and  survivability 
needs  and  what  is  actually  affordable  and  technically  possible  within  program  funding, 
risk,  and  time  constraints.  ORD  reliability  requirements  are  developed  in  accordance 
with  AR  71-9  [Ref.  9].  Three  key  elements  combine  to  define  overall  reliability 
performance  requirements:  list  them  briefly. 
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2.  Program  Manager’s  Role  in  the  Reliability  Requirements  Process 

Once  the  overall  system  requirement  is  established  in  the  ORD,  the  DoD 

delegates  the  responsibility  for  implementing  the  reliability  requirement  to  the  Material 

Developer  (MATDEV),  who  is  usually  a  Program  Manager  (PM).  The  PM  is  responsible 

for  the  system  acquisition,  including  the  reliability  requirement.  DoD  5000. 2-R  states 

that  the  “the  PM  has  the  task  to  ensure  that  reliability,  maintainability,  and  availability 

activities  are  established  early  in  the  acquisition  cycle  to  assure  meeting  operational 

requirements  and  reduced  life-cycle  ownership  cost."  [Ref.  10]  The  Army  further 

requires  the  PM  to  develop  and  implement  an  effective  Reliability  &  Maintainability 

(R&M)  program  through  AR  70-1,  which  states: 

[An]  R&M  program  will  be  tailored  in  scope  and  content  and  be  designed 
to  ensure  that  the  user  operational  reliability  requirements  will  be  met  at 
confidence  levels  established  by  the  user. 

Finally,  DA  PAM  70-3  guidance  covers  aspects  of  R&M  Requirements,  R&M 
Management,  R&M  Engineering  and  Design,  R&M  Testing,  and  R&M  and  Assessment 
Integrated  Process  Team  (IPT)  procedures.  [Ref.  1 1]  The  PM  must  take  a  proactive  role 
in  establishing  systems  reliability,  starting  at  program  initiation  and  continuing  through 
the  entire  program  life  cycle. 

3.  Contracting  for  Reliability 

It  is  the  now  the  PM’s  responsibility  to  insure  that  reliability  objectives  are 
translated  into  quantifiable  and  verifiable  contractual  terms.  Prior  to  the  advent  of 
military  specifications  and  standards  reform  in  1994,  the  work  requirements  for  reliability 
engineering  were  usually  described  in  a  Statement  of  Work  (SOW)  task  that  required 
compliance  with  MIL-STD  785,  "Reliability  Program  for  Systems  and  Equipment 
Development  and  Production.”  [Ref.  12]  In  February  1996,  Gil  Decker,  the  Army 
Acquisition  Executive  at  the  time,  issued  policy  on  incorporating  a  performance-based 
approach  to  Reliability  in  Requests  for  Proposals  (RFPs).  A  key  change  was  that  no 
“how  to”  reliability  standardization  documents  were  to  be  used.  The  policy  stated: 

Reliability  requirements  should  be  included  in  RFPs  by  specifying: 

(1)  quantified  reliability  requirements  and  allowable  uncertainties,  (2) 
failure  definitions  and  thresholds,  (3)  life-cycle  usage  conditions.  [Ref.  13] 
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Decker’s  policy  was  institutionalized  in  the  update  to  AR  70-1  in  January  1998. 
AR  70-1  clarified  several  points  of  the  AAE  memo. 

Reliability  parameters  expressed  by  operational  users,  and  ones  specified  in 
contractual  documents,  take  many  different  forms.  User  requirements  are  generally 
expressed  in  a  variety  of  forms  that  include  combinations  of  mission  and  logistics 
reliability,  or  they  may  combine  reliability  with  maintainability  in  the  form  of 
availability.  Conversion  from  commonly  used  operational  terms,  such  as  mean-time- 
between-maintenance  (MTBM)  and  mean-time-between-critical-failure  (MTBCF),  must 
be  made  to  enable  translation  to  parameters  that  can  be  specified  in  contracts  and  verified 
in  testing. 

In  order  to  verify  that  a  contractor  will  achieve  the  system  reliability  requirement, 
the  PM  will  also  include  contract  provisions  that  ensure  access  to  sufficient  information 
for  evaluating  source  data,  models,  reasonableness  of  modeling  assumptions,  methods, 
results,  risks,  and  uncertainties.  [Ref.  14]  Also,  the  test  program  must  be  structured  so 
that  the  reliability  requirements  can  be  demonstrated  prior  to  full-rate  production.  One  of 
the  cardinal  rules  of  testing  is  to  verify  contractor  claims  of  reliability  as  early  as 
possible.  [Ref.  15]  The  infonnation  used  to  estimate  or  detennine  progress  toward 
meeting  R&M  requirements  must  be  based  on  testing  in  accordance  with  the  system’s 
expected  mission  profile  and  operating  environment.  Reliability-growth  projections  or 
other  prediction  methods  are  sometimes  used  in  determining  that  contractual  reliability 
requirements  have  been  met. 

4.  Reliability  Allocation 

After  an  acceptable  requirement  has  been  established  for  the  system,  it  must  be 
allocated  among  the  various  subsystems.  Reliability  allocation  involves  setting  reliability 
objectives  for  components  or  subsystems  in  order  to  meet  a  system-level  reliability 
objective.  Reliability  allocation  should  occur  in  the  initial  stages  of  design.  A 
knowledge  of  technical  capability  and  cost  trade-off  considerations  will  help  derive 
optimum  allocations,  but  the  simplest  method  for  allocating  reliability  is  to  distribute  the 
objective  uniformly  among  all  subsystems.  For  example,  consider  a  simple  system 
composed  of  two  subsystems:  subsystem  A  and  subsystem  B.  Each  subsystem  is 
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composed  of  assemblies  (assembly  A  through  assembly  P).  Each  assembly  is  composed 
of  components  (component  1  through  component  10).  The  reliability  of  the  system  is 
based  on  the  reliability  characteristics  of  the  subsystems;  the  reliabilities  of  the 
subsystems  are  based  on  the  reliability  characteristics  of  the  assemblies;  and  so  forth.  At 
the  lowest  level  (i.e.,  the  component  level),  reliability  is  assumed  or  predicted.  [Ref.  16] 
Figure  2  displays  reliability  block  diagrams  for  this  example — created  using  a  sub¬ 
diagram  approach — and  parameters  for  each  component. 

RELIABILITY  ALLOCATION 

PURPOSE: 

-  IN-DEPTH  ANALYSIS  ENSURING  THAT  REQUIREMENTS  ARE  ALLOCATED 
FROM  THE  SYSTEM  LEVEL  TO  THE  REQUIRED  SUB-LEVEL 


RELIABILITY  =  0.999  PER  COMPONENT 


Figure  2.  System  Reliability  Allocation  Diagram  Example 


5.  Causes  and  Impacts  of  Establishing  a  Flawed  Requirement 

One  of  the  greatest  challenges  designing  a  weapon  system  is  establishing  an 
acceptable  reliability  requirement.  Sometimes,  established  reliability  requirements  are 
not  realistic.  Although  the  PM  is  supposed  to  be  involved  in  generating  requirements,  in 
reality,  R&M  Managers  from  the  program  office  are  given  little  voice  during  this  part  of 
the  process.  The  checks  and  balances  to  ensure  a  proper  requirement  were  either 
removed  as  part  of  acquisition  streamlining  or  are  simply  not  in  place.  [Ref.  3] 

Program  offices  must  also  deal  with  unrealistic  contractor  estimates.  During 
initial  acquisition,  the  contractor  engineers  provide  reliability  estimates  that  are  critical  in 
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establishing  sparing  levels  and  technical  support  criteria.  Experience  has  shown  that 
contractors,  who  are  under  pressure  to  win  and  maintain  contracts,  use  optimistic 
reliability  predictions.  [Ref.  17]  The  null  hypothesis  of  having  the  contractor  verify 
reliability  estimates  as  accurate  computations  is  seldom  enforced.  Most  weapon  systems 
are  cost-  and  schedule-driven  and  do  not  incorporate  the  time  or  resources  necessary  to 
perform  reliability  tests  on  critical  sub-systems.  [Ref.  3,  Ref.  17] 

The  impacts  of  a  flawed  requirement  are  two-fold.  Not  only  will  the  system  have 
difficulty  meeting  the  requirement  during  testing,  but  the  development  of  a  sustainment 
package  based  upon  a  flawed  reliability  factor  also  will  produce  sparing  shortages,  test 
equipment  shortages,  and  decreased  readiness.  [Ref.  15] 

E.  HIGH  RELIABILITY  AND  THE  LINK  TO  LOGISTICS  AND  LILE-CYCLE 

SUPPORT  COSTS 

The  failure  of  a  system  to  achieve  high  reliability  levels  has  a  severe  impact  on 
operation  and  support  (O&S)  costs  and  operational  availability  (Ao).  The  impact  of 
reliability  on  O&S  costs  is  attributable  to  the  high  costs  associated  with  acquiring  repair 
and  replacement  parts.  High  reliability  is  fundamental  to  cost-effective  and  efficient 
logistics;  it  means  less  support  and  training  and  fewer  inventory  costs  and  personnel. 
According  to  an  analysis  of  the  Army  Comanche  Program,  missing  predicted  reliability 
goals  by  just  one  percent  could  result  in  an  increase  in  O&S  costs  of  over  $75  million. 
[Ref.  2]  Figure  3  demonstrates  the  huge  O&S  costs  associated  with  maintaining  some 
other  Anny  weapon  systems.  As  the  chart  shows,  the  potential  impact  on  lowering  O&S 
costs  through  increasing  reliability  is  enonnous. 
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Wholesale  $  Demands  Per  Year 


Figure  3. 


Cost  to  Support  Current  Reliability  Levels  [From  Ref.  5] 


The  impact  of  reliability  on  readiness  also  can  be  significant.  In  the  Comanche 
Program,  for  each  five  percent  below  the  predicted  reliability  level  for  peacetime 
operations,  the  Ao  decreases  by  one  percent.  [Ref.  2]  Figure  4  demonstrates  the  impact 
of  reliability  on  Army  tank  availability.  The  impact  is  significant,  as  45  percent  of  the 
tanks  were  down  at  one  point  during  the  gunnery  firing  exercise. 
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Day 


Figure  4.  Battalion’s  Tank  Fleet  Hit  45  Percent  Down  During  A  Gunnery  [From  Ref.  5] 


Reliability  also  affects  troop  morale.  Because  low  reliability  causes  a  high  failure 
rate,  it  makes  unrealistic  demands  on  maintenance  personnel.  Repairs  and  maintenance 
may  have  to  be  perfonned  very  quickly  at  any  time,  day  or  night,  in  order  to  meet 
operational  commitments.  In  such  cases,  personnel  must  continue  working  until  the  job 
is  done,  regardless  of  how  much  time  it  takes.  An  August  1999  GAO  survey  of  active- 
duty  personnel  showed  that  a  significant  portion  of  aviation  maintenance  personnel 
worked  more  than  50  hours  a  week,  and  that  some  worked  70  hours  a  week.  The  GAO 
reported  that  the  majority  of  factors  cited  by  military  personnel  as  sources  of 
dissatisfaction  and  reasons  for  leaving  the  military  were  work-related,  such  as  the  lack  of 
parts  and  materials  to  successfully  complete  daily  job  requirements.  One  example  is  an 
aircraft  wing  with  a  status  report  showing  that  six  out  of  28  aircraft  downed  for  parts  had 
not  flown  for  37  days  or  more.  One  of  these  aircraft  had  not  flown  for  more  than  300 
days  and,  according  to  the  Maintenance  and  Material  Control  Officer,  was  missing  111 
parts.  As  a  result,  the  Navy  had  been  unable  to  use  this  multimillion-dollar  asset  for 
almost  a  year.  [Ref.  18] 

When  faced  with  supporting  maintenance-intensive  weapon  systems,  technicians 
and  logisticians  are  initiated  into  a  culture  of  learning  to  do  more  with  less  and 
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performing  the  mission  at  all  cost.  Failure  to  perform  a  mission  successfully  is  deemed 
an  unacceptable  outcome.  Rather  than  fail,  technicians  (and  commands)  resort  to 
shortcuts,  work-arounds,  and  misrepresentations  of  readiness  to  feign  the  appearance  of 
mission  readiness.  [Ref.  19]  To  the  extent  that  low  reliability  causes  failures  that 
contribute  to  extensive  overtime,  morale  and  retention  will  be  adversely  affected,  and 
additional  costs  may  be  incurred  in  recruiting  and  training  new  personnel 
F.  BENEFIT  OF  “DESIGNING  IN”  RELIABILITY 

Although  the  reliability  development  process  consists  of  many  elements  (e.g.,  a 
PM&P  Program,  Test  and  Evaluation,  Design,  Design  Interface,  utilization,  etc.),  the 
benefit  of  “designing  in”  reliability  cannot  be  understated.  “Designing  in”  reliability 
implies  perfonning  up-front  analyses  during  the  design  phase  so  that  the  inherent 
reliability  of  the  system  is  as  high  as  possible.  As  with  anything  else,  being  proactive 
with  reliability  early  in  the  lifecycle  of  a  system  is  more  cost-effective  than  dealing  with 
potential  schedule  delays  and  unexpected  costs  of  failing  a  test  later,  only  to  have  to 
redesign  and  test  yet  again  until  the  problem  is  fixed.  It  is  critical  to  make  reliability  part 
of  the  early  design  because  the  decisions  made  during  this  period  have  such  a  great 
impact  on  the  Life  Cycle  Cost  (LCC).  The  chance  to  influence  LCC  cost  reductions 
grows  smaller  when  projects  are  converted  into  bricks  and  mortar,  as  Figure  5  illustrates. 
Making  major  changes  in  LCC  after  projects  are  turned  over  to  production  is  not  possible 
because  the  die  has  been  cast.  [Ref.  20] 
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Of  Life  Of  Life 

Life  Cycle  Span 


Figure  5.  %LCC  vs.  Life  Cycle  Span  [From  Ref.  20] 


In  the  past,  the  Army  has  relied  on  costly  test-analyze-fix  (i.e.,  reliability  growth) 
programs  in  order  to  "grow"  reliability  until  requirements  were  met.  The  decision  to 
“design  in”  reliability  depended  on  the  individual  program.  There  are,  however, 
significant  benefits  to  “designing  in”  reliability.  Designing  in  reliability  will  achieve 
higher  system  reliability  up-front,  thus  reducing  unforeseen  and  costly  failures  during 
later  phases  in  a  program.  Highly  reliable  components  will  significantly  reduce  a 
system's  O&S  costs  and,  thus,  reduce  the  expensive  process  of  reliability  growth  and  the 
resulting  project  delay  costs.  It  is  crucial  to  emphasize,  early  on,  the  use  of  proper  design 
tools  and  activities  to  “build  in”  reliability  up-front,  rather  than  relying  on  extensive 
“back  end”  testing  and  validation.  [Ref.  21] 
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In  a  briefing  on  22  February,  2001,  Tom  Edwards,  Deputy  to  Commanding 
General  Combined  Arms  Support  Command  (CASCOM),  told  the  International  Society 
of  Logistics  that  reliability  can  be  significantly  improved  by  focusing  on  two  main  areas: 
1)  designing  in  reliability  up-front,  and  2)  validating  that  we  have  reliable  systems  and 
components.  This  highlights  the  concept  that  reliability  can  be  significantly  improved  by 
design  early  in  the  development  process. 

If  a  fielded  weapon  system  is  to  have  a  high  system  readiness  objective — either 
operational  availability  or  probability  of  mission  success — it  must  have  high  reliability. 
It  generally  is  understood  that  to  have  a  high  level  of  readiness,  one  must  have  a  reliable 
system.  [Ref.  8]  For  a  system  to  be  reliable  in  the  field,  it  must  have  good  inherent 
design  reliability  that  starts  with  the  original  design  concept  and  continues  throughout  the 
development  of  the  design.  To  reduce  the  risk  of  creating  an  unreliable  system,  the 
design  should  be  required  to  follow  certain  reliability  and  maintainability  practices  and 
guidelines.  It  is  essential  that  these  practices  and  guidelines  begin  early  in  the  design 
concept  because  poor  or  marginal  reliability  is  often  difficult  to  detect  before  the  design 
effort  is  completed.  The  following  section  presents  the  benefits  of  using  highly  reliable 
parts,  materials,  and  processes  to  achieve  a  high  inherent  reliability. 

G.  USING  A  PM&P  PROGRAM  TO  “DESIGN-IN”  RELIABILITY 

Simply  stated,  highly  reliable  parts,  materials,  and  processes  are  the  building 
blocks  of  a  highly  reliable  weapon  system.  While  the  system  design  approach  is  largely 
responsible  for  the  weapon  system’s  functional  performance,  the  PM&P  Program 
approach  is  responsible  for  a  large  part  of  the  weapon  system’s  reliability  performance. 
Therefore,  to  insure  that  the  reliability  perfonnance  is  being  met,  one  has  to  evaluate  the 
PM&P  Program  requirements,  their  implementation  and  verification,  and  the 
requirements’  translation  to  the  lowest  part,  material  and  process  applied  in  the  system. 

The  goal  of  a  PM&P  Program  is  the  selection  of  parts,  materials,  and  processes 
that  can  withstand  the  manufacturing  environment  and  reliably  perform  the  needed 
function  for  the  design  life  of  the  product  in  the  intended  environment.  Selection 
involves  designing  with  parts  and  materials  that  have  known  reliability,  superior  life, 
durability,  and  corrosion  resistance.  Application  involves  using  a  PM&P  Program  to 
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ensure  that  field  environment  stresses  are  less  than  the  parts’  and  materials’  strengths. 
Individual  part,  material,  and  process  selection  criteria  and  a  PM&P  Program  should  be 
used  in  the  preliminary  design  to  ensure  that  designers  are  provided  preferred  parts 
selection  lists  before  design  begins  and  that  they  continue  to  use  only  approved  parts, 
materials,  and  processes  in  the  system  design.  Examples  of  the  adverse  effects  from 
designing  without  a  PM&P  Program  include  unknown  failure  mechanisms,  inappropriate 
corrective  actions,  inappropriate  design  rules,  low  system  reliability,  etc.  [Ref.  6] 

The  following  case  of  microcircuits  in  a  tactical  missile  offers  an  example  of  how 
a  PM&P  Program  affects  system-level  cost  and  reliability.  Suppose  that  the  cost  of  the 
missile  is  $100,000,  and  the  missile  contains  70  microcircuits,  each  of  which  can  cause  a 
mission-critical  failure.  If  2000  missiles  are  produced,  then  there  will  be  a  total  of 
140,000  microcircuits.  Now  further  assume  that  each  microcircuit  has  0.9989  reliability, 
and  the  reliability  is  allocated  serially  (as  in  Figure  2).  This  means  that,  on  average,  there 
will  be  148  failed  missiles  and  a  lost  value  of  $14,800,000.  Now  assume  that  there  is  a 
reliability  test  that  can  ensure  100  percent  reliability  and  that  this  test  will  be  performed 
on  each  lot  of  microcircuits  at  $  12,000/lot.  The  total  cost  of  the  test  would  be  $924,000 
(with  tests  for  seven  additional  lots).  The  lost  value  exceeds  the  test  cost  by  — 16:1  and 
would  likely  justify  the  test  cost  to  avoid  the  losses.  This,  of  course,  ignores  the 
potentially  life-threatening  consequences  of  a  tactical  missile  failure.  [Ref.  22] 

H.  FORMER  AND  CURRENT  DOD  PM&P  PROGRAM  POLICIES  AND 

GUIDANCE 

The  PM&P  program  is  founded  on  DoD  4120.24-M  Defense  Standardization 
Program  Policies  and  Procedures,  which  establishes  the  current  policies  and  procedures 
to  achieve  the  objectives  required  by  10  U.S.C. 245 1-2457,  DoD  Directive  5000.1,  and 
DoD  5000.2-R.  Under  the  DoD’s  performance-based  acquisition  policies,  it  is  primarily 
the  contractor’s  responsibility  to  recommend  the  use  of  materials,  parts,  components,  and 
other  items  needed  to  meet  performance  requirements  and  satisfy  other  program 
elements,  such  as  parts  management  and  logistics  support.  Paragraph  C3.2.4  of  DoD 
4120.24-M  contains  the  following  mandatory  requirement: 
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Program  offices  shall  ensure  that  a  parts 
management  process  is  used  to  reduce  the  proliferation  of 
parts  and  associated  documentation  and  promote  the  use  of 
parts  with  acceptable  performance,  quality,  and  reliability. 

[Ref.  23] 

Prior  to  acquisition  reform,  detailed  parts  control  program  requirements  for  use  on 
newly  designed  and/or  modified  equipment  were  typically  invoked  in  contracts  through 
Task  207  of  MIL-STD-785.  Task  207  required  the  establishment  of  a  parts  control 
program  in  accordance  with  MIL-STD-965.  Figure  6  illustrates  the  relationship  of  MIL- 
STD-965  to  MIL-STD-785  and  other  parts  and  material  specifications.  One  of  the 
provisions  of  MIL-STD-965  required  contractors  to  submit  parts  lists  to  the  Government 
for  approval.  Another  provision  was  a  part  selection  order  of  preference,  in  which 
military  specification  parts  were  considered  first,  and  commercial  parts  could  be 
considered  only  when  no  military  part  was  available. 
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Reform  [From  Ref.  24] 


In  order  to  be  consistent  with  acquisition  reform  initiatives,  MIL-STD-965  was 
converted  to  MIL-HDBK-965  in  September  1996.  In  October  2000,  MIL-HDBK-512 
was  released,  and  MIL-HDBK-965  was  cancelled.  MIL-HDBK-512  now  provides  the 
current  guidance  (guidance  only —  if  it  is  cited  as  a  requirement,  the  contractor  does  not 
have  to  comply)  for  implementing  an  effective  parts  management  program  on 
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DoD/Industry  acquisitions.  MIL-HDBK-512  supports  the  acquisition  strategies  and 
systems  engineering  practices  of  DoD  4120.24-M  and  DoD  5000.2-R.  It  also  provides 
performance-based  parts  management  process  guidance,  including  the  objective  to 
enhance  the  reliability  of  end-item  performance.  MIL-HDBK-512  no  longer  requires 
contractors  to  submit  parts  to  the  Government  for  approval;  nor  does  it  require  the 
preference  of  military  specification  parts  over  commercial  parts.  The  management 
elements  of  MIL-HDBK-512  include:  [Ref.  7] 

1 .  Parts  selection  baseline.  This  is  a  parts  selection  list  or  database  that  should  be 
maintained  in  order  to  give  to  designers  and  subcontractors  the  list  of  preferred 
parts  and  materials. 

2.  Parts  selection  and  authorization  process.  These  are  the  procedures  for 
authorizing  new  parts  to  the  preferred  parts  selection  list.  The  procedures  will 
identify  the  IPT  or  organization  responsible  for  authorizing  parts  for  use  and 
the  structure  and  membership  of  the  IPT.  The  procedures  will  also  include 
criteria  used  to  ensure  suitability  of  parts  intended  for  use  in  the  required 
application;  order  of  preference  used  in  considering  new  parts,  and  procedures 
for  notifying  associated  disciplines  (inventory,  purchasing,  quality  assurance) 
in  case  authorization  of  a  new  part  should  be  included. 

3.  Obsolescence  management.  This  includes  procedures  for  obsolescence 
management,  which  include  proactive  obsolescence  forecasting  for  applicable 
part  types  (e.g.,  microcircuits)  and  plans  for  reacting  and  achieving  solutions  to 
obsolescence  impacts  as  they  occur  and  affect  the  program. 

4.  Subcontractor  management.  This  element  addresses  contractor  procedures  for 
establishing  and  maintaining  subcontractor  IPT  participation  to  ensure 
satisfaction  of  the  PM&P  Program  objectives. 

5.  Part  and  supplier  quality.  This  management  element  provides  the  requirements 
for  assessing  part  suppliers  and  part  quality,  such  as  statistical  process  control 
data,  audits,  past  perfonnance,  etc. 

6.  Part-level  documentation  requirements.  Part-level  documentation  procedures 
need  to  be  consistent  with  both  the  program’s  logistic  strategies  and  its  need  for 
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performance  and  re-procurement  documentation  at  the  intended  level  of  logistic 
support. 

7.  Substitute  and  alternate  part  procedures.  This  section  addresses  the  process  for 
the  management  and  documentation  of  parts,  other  than  those  on  an  as-built  or 
as-designed  parts  list.  The  program  needs  to  be  consistent  with  the  intent  and 
application  of  other  disciplines  (e.g.,  reliability,  configuration  management, 
quality,  logistics,  etc.). 

8.  Customer-contractor  teaming.  This  part  of  the  management  plan  addresses 
how  customer  teaming  is  provided  so  that  process  insight  and  program 
verification  (i.e.,  IPT  participation,  technical  interchange  meetings,  as-built, 
and  other  parts  lists)  are  allowed. 

I.  COMMERCIAL  PM&P  PROGRAM  PRACTICES 

The  American  Institute  of  Aeronautics  and  Astronautics  (AIAA)  has  established 
AIAA  R-100  [Ref.  25],  a  document  that  is  a  collection  of  recommended  practices  for 
managing  parts  and  materials.  The  AIAA  standard  establishes  a  management  approach 
that  is  consistent  with  the  current  business  environment.  The  document  strategy  is  to 
manage  risk  up-front  by  selecting  the  right  part  for  the  intent  application. 

The  standard  emphasizes  that  a  PM&P  Program  will  deal  pro-actively  with  parts 
and  materials  management  issues  and  provide  guidance  for  developing  comprehensive 
strategies  to  manage  cost  and  schedule  risk  via  an  IPT  process.  Figure  7  depicts  the  IPT 
process.  The  design  process  includes  design  margins,  life  cycle  cost,  technology 
insertion  strategy,  technical  support,  parts  selection,  and  validation,  all  of  which  are 
addressed  concurrently  rather  than  sequentially.  [Ref.  25] 
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Figure  7.  PM&P  Management  IPT  Overview  [From  Ref.  25] 

AIAA  R-100  also  establishes  a  design  process.  The  flow  diagram  (Figure  7) 
illustrates  that  the  interrelationships  between  the  critical  key  elements  of  AIAA  R-100 
must  be  addressed  concurrently  to  achieve  the  “best  practice”  selection  of  parts, 
materials,  and  documentation  required  for  the  design.  Figure  8  is  important  because  it 
shows  not  only  the  key  elements  that  must  be  considered,  but  also  that  the  objective  of 
this  design  process  is  to  evaluate  inputs  from  ah  the  key  elements  concurrently  and  to 
then  select  the  parts  and  materials  that  satisfy  the  product  requirements.  Figure  8  is 
helpful  to  the  DoD  because  it  demonstrates  that  there  is  a  commercial  “best  practice” 
document  with  a  parts  and  materials  design  approach. 
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Figure  8.  Commercial  Best  Practice  Part  Design  Process  [From  Ref.  25] 

J.  SYSTEM-  LEVEL  RELIABILITY  AND  ITS  IMPACT  ON  THE  PM&P 

PROGRAM 

Section  D  mentioned  some  of  the  impacts  of  selecting  a  flawed  system-level 
reliability  requirement.  The  impacts  of  this  requirement  are  far-reaching,  even  down  to 
the  individual  part  level.  Once  the  reliability  requirement  has  been  established,  it  will  be 
allocated  among  the  various  subsystems,  units,  assemblies,  and  so  on.  For  a  system  with 
a  high  reliability  requirement,  this  means  that  the  parts  that  make  up  the  system  will  need 
to  have  even  higher  reliability. 

For  example,  consider  a  system  requiring  90  percent  reliability.  If  the  system 
were  modeled  with  six  components,  each  component  would  require  a  reliability  of  greater 
than  98  percent,  if  evenly  allocated.  At  this  high  reliability  level,  the  system  becomes 
very  sensitive  to  even  one  component  not  meeting  its  allocation 
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An  example  of  the  effect  at  the  part  level  is  the  U.S.  Army  JAVELIN  Missile. 
The  guidance  electronics  circuit  assemblies  have  a  reliability  allocation  of  about  99 
percent,  which  includes  about  120  microcircuits  and  semiconductors.  Therefore,  even  if 
everything  else  is  perfect  (no  solder  joint,  printed  wiring  board,  or  passive  part  failures), 
each  microcircuit’s  and  semiconductor’s  part  reliability  must  exceed  99.99  percent.  [Ref. 
22]  Now  consider  a  system  such  as  an  airplane  or  helicopter,  which  has  thousands 
instead  of  hundreds  of  such  parts.  Part  reliability  must  be  higher  than  system  reliability, 
and  even  much  higher  depending  on  the  number  of  parts;  and  failure  to  meet  allocation 
on  even  one  part  would  reduce  the  overall  system  reliability.  The  system-level  reliability 
requirement  is  critical  because  once  it  gets  allocated  to  the  lower  levels,  it  can  easily 
drive  the  design  and  test  at  the  lower  levels  of  assembly  (such  as  the  individual  piece 
parts)  to  unreasonable  levels. 

K.  CHAPTER  SUMMARY 

In  this  chapter,  the  researcher  described,  first,  the  current  reliability  trend  in  the 
Anny  and,  then,  the  PM&P  Program.  Background  on  how  a  reliability  requirement  is 
established  on  a  weapon  system  included  the  origination  of  the  requirement,  how  it  is 
allocated,  and  the  impact  of  establishing  a  flawed  requirement.  The  relationship  of 
reliability  to  logistics  and  life  cycle  costs  was  demonstrated.  The  benefit  of  “designing 
in”  reliability  was  explained,  along  with  the  impact  that  system-level  reliability  can  have 
on  the  individual  parts  and  materials.  The  chapter  covered  former  and  current  DoD 
policies  and  guidance,  as  well  as  commercial  PM&P  initiatives. 
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III.  DATA 


A.  INTRODUCTION 

This  chapter  identifies  and  discusses  a  variety  of  issues,  common  practices, 
concerns,  and  real-world  experiences  of  subject  matter  experts  (SMEs)  who  work  on 
PM&P  Programs.  Data  are  presented  from  SMEs  who  work  on  programs  that  include 
missiles,  aircraft,  and  space  systems  in  various  stages  of  development.  The  data  were 
gathered  through  telephone  interviews,  using  questions  based  on  the  literature  review  and 
the  background  research  on  PM&P  Programs  described  in  Chapter  II.  The  questions 
were  designed  to  draw  out  the  experiences  and  practices  of  each  SME  on  managing 
PM&P  Programs. 

This  chapter  is  organized  as  follows:  first,  it  discusses  the  general  methodology 
and  the  process  used  to  conduct  the  interviews;  second,  the  chapter  presents  the  interview 
question  responses,  grouped  by  common  themes,  and,  where  appropriate,  provides  them 
in  tabular  form;  and,  finally,  it  presents  conclusions.  Note  that,  with  the  exception  of  the 
definitions  in  Table  3,  the  source  for  the  tables  found  in  this  chapter  is  the  author,  who 
based  them  on  responses  to  the  interviews. 

B.  METHODOLOGY 

1.  Overview 

This  thesis  focuses  on  optimizing  the  use  of  highly  reliable  parts,  materials,  and 

processes — a  key  element  in  maximizing  weapon  system  reliability.  The  objective  is  to 

develop  a  set  of  heuristics  to  assist  future  Program  and  Logistics  Managers.  The  method 

for  accomplishing  this  objective  is  to  analyze  the  methods  and  beliefs  of  those  who  are 

considered  subject  matter  experts  (SMEs)  in  the  field.  Analysis  of  the  tactics  and 

strategies  that  can  be  used  to  maximize  the  inherent  reliabilities  of  system  components 

will  not  only  help  future  Program  and  Logistics  Managers,  but  will  also  assist  the  DoD  in 

achieving  reliability  goals  that  remain  elusive.  Since  the  primary  research  question 

addresses  “what”  tactics  and  strategies  can  be  used,  the  best  way  to  answer  the  question 

may  be  through  the  use  of  in-depth  cases.  However,  since  in-depth  cases  were  not 

practical  for  this  thesis  due  to  resource  constraints,  telephone  interviews  with  SMEs,  the 
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individuals  responsible  for  implementing  the  PM&P  Program,  were  conducted  instead. 
[Ref.  26]  The  purpose  of  the  interviews  was  to  gather  data  in  order  to  identify  common 
themes. 

2.  Development  of  Interview  Questions 

The  interview  questions  evolved  from  the  secondary  research  questions.  The 
questions  and  their  relationship  to  the  research  questions  are  depicted  in  the  following 
matrix.  (Table  1) 
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Secondary  Research  Questions 

Related  Interview  Questions 

1 .  What  technology,  tools,  and  techniques 
are  available  to  ensure  the  proper  selection 
of  reliable  parts  and  materials? 

a.  What  design  tools  can  be  employed  to 
ensure  that  reliability  is  “built  in”  early  in 
the  design  of  a  weapons  system? 

b.  What  methodologies  and  techniques  are 
used  to  determine  parts,  materials,  and 
processes  compliance? 

c.  What  methodologies  and  techniques 
have  succeeded,  and  do  you  have  any 
examples? 

d.  What  methodologies  and  techniques 
have  not  succeeded,  and  do  you  have  any 
examples? 

2.  How  has  acquisition  reform  changed  the 
way  a  PM&P  Program  is  managed? 

a.  How  was  a  PM&P  Program  managed 
prior  to  acquisition  reform? 

b.  How  did  acquisition  reform  change  the 
way  the  PM&P  Program  is  managed? 

c.  How  did  PM&P  Program  risks  increase 
or  decrease  with  acquisition  refonn? 

3.  To  what  extent  does  commercial 
industry  differ  in  its  approach  to  part  and 
material  selection,  and  can  the  Military 
leverage  these  best  practices  to  improve 
performance  in  military  systems? 

a.  What  methodologies  and  techniques 
does  commercial  industry  use  to  manage 
the  PM&P  Program? 

b.  What  commercial  industry  techniques 
can  DoD  adopt? 

4.  How  can  a  Program  and  Logistics 
Manager  mitigate  part  and  material  risks? 

a.  What  are  the  top  three  things  a  Program 
Manager  can  do  to  mitigate  part,  material, 
and  process  risk  and  maximize  reliability  in 
the  design? 

b.  How  much  and  what  kind  of  input  does 
the  PM  get  from  the  Logistics  Manager  on 
PM&P  Program  issues? 

c.  Could  the  level  of  input  be  improved? 
Why  or  why  not? 

d.  If  there  is  input  from  the  Logistics 
Manager,  is  it  useful? 

e.  Are  there  any  examples  (good  or  bad)  of 
this  interaction? 

Table  1.  Relationship  Between  Research  and  Interview  Questions 


3.  Choice  of  Study  Subjects 

Interview  subjects  were  chosen  based  on  convenience  sampling.  Although  this 

technique  may  not  be  representative  of  the  population  as  a  whole,  the  information  can 
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still  provide  some  significant  insights  and  be  a  good  source  of  data.  [Ref.  27]  Interviews 
were  conducted  with  individuals  from  the  Anny  (U.S.  Army  Aviation  and  Missile 
Command),  Navy  (Naval  Air  Systems  Command),  and  Air  Force  (Space  &  Missile 
Systems  Center)  and  with  contractors  with  whom  they  interface.  Due  to  their  high  cost 
and  the  safety-related  nature  of  their  application,  these  particular  systems  require  a  high 
degree  of  reliability.  Each  SME  has  a  minimum  of  ten  years’  experience  working  with 
PM&P  Programs  on  military  weapon  systems.  A  total  of  1 1  interviews  were  conducted. 
This  base  of  subject  matter  experts  should  provide  a  fairly  representative  cross-section  of 
experiences  in  utilizing  highly  reliable  parts,  materials,  and  processes  in  weapon  systems. 

4.  Interviews 

Interview  questions  were  distributed  to  each  SME  via  email,  along  with 
information  regarding  the  objectives  of  the  interview.  This  email  was  then  followed  up 
with  a  telephone  interview.  Since  the  interview  subjects  were  located  throughout  the 
country,  in-person  interviews  would  have  been  neither  cost-effective  nor  timely.  [Ref.  26] 
In  order  to  ensure  that  the  data  were  valid,  the  interview  subjects  had  full  editorial 
control.  If  the  answers  were  recorded  incorrectly,  the  SME  was  given  full  opportunity  to 
make  corrections,  or  else  the  data  were  not  used. 

Questions  were  written  in  a  manner  to  avoid  justifications — that  is,  they  asked 
"how"  instead  of  “why.”  Leading  questions  and  “yes  or  no”  questions  were  avoided,  as 
well.  The  goal  was  to  get  the  subjects  to  express  their  ideas  and  thoughts  about  the 
issues.  Table  2  summarizes  the  steps  used  in  conducting  the  study. 


1. 

Identify  and  analyze  the  primary  research  question. 

2. 

Conduct  a  review  of  reliability  and  PM&P  Program  literature. 

3. 

Develop  interview  questions  for  subjects  to  obtain  data  to  answer  primary  and 
secondary  research  questions. 

4. 

Conduct  a  review  of  personnel  who  have  extensive  experience  with  PM&P 
Programs  and  issues. 

5. 

Arrange  for  and  conduct  telephone  interviews  with  subjects. 

6. 

Analyze  subjects’  responses  for  common  issues. 

7. 

Compare  subjects’  responses. 

8. 

Develop  generalizations  and  summarize  data. 

9. 

Develop  a  set  of  critical  strategies  and  tools  to  aid  future  PMs  and  LMs. 

Table  2.  Summary  of  Methodology 
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C.  TECHNOLOGY,  TOOLS,  AND  TECHNIQUES 


The  first  group  of  questions  focused  on  the  technology  tools  and  techniques  that 
are  used  to  select  and  control  the  parts,  materials,  and  processes;  how  compliance  is 
verified;  and  what  methods  were  successful  and  unsuccessful.  The  first  question  asked 
about  the  design  tools  that  can  be  employed  to  ensure  that  reliability  is  “built  in”  early  in 
the  design  of  a  weapons  system.  The  SMEs’  answers  and  their  definitions  are  shown  in 
the  Table  3  below. 

Reliability  Prediction  Analysis  -  is  simply  the  analysis  of  parts  and  components  in 
an  effort  to  predict  the  rate  at  which  an  item  will  fail.  A  reliability  prediction  is  one 
of  the  most  common  forms  of  reliability  analyses. 

Circuit  Tolerance  Analysis  -  is  a  technique,  which,  by  accounting  for  component 
variability,  determines  circuit  performance  under  a  worst-case  scenario,  i.e.,  under 
extreme  environmental  or  operating  conditions.  This  analysis  is  particularly  cost- 
effective  in  safety-critical  applications  and  for  products  expected  to  operate  in  severe 
environments. 

Sneak  Circuit  Analysis  -  detects  design  flaws  that  escape  reliability  testing  and 
impacts  system  performance  by  activating  unwanted  functions  or  impeding  desired 
functions.  It  is  crucial  that  sneak  circuits  are  found  as  early  as  possible.  Failure  to 
do  so  can  lead  to  costly  late  redesign  or  safety-related  failures. 

Physics  of  Failure  (PoF)  Tools  -  is  an  approach  that  uses  knowledge  of  root-cause 
failure  processes  to  prevent  product  failures  through  robust  design  and  manufacturing 
practices.  The  PoF  approach  proactively  incorporates  reliability  into  the  design 
process  by  establishing  a  scientific  basis  for  evaluating  new  materials,  structures,  and 
electronics  technologies.  The  PoF  approach  encourages  innovative,  cost-effective 
design  through  the  use  of  realistic  reliability  assessment. 

Part  Derating  -  is  the  practice  of  limiting  electrical,  thermal  and  mechanical  stresses 
on  parts  to  levels  below  their  specified  ratings.  If  a  part  or  material  is  expected  to 
perform  reliably,  it  must  incorporate  a  conservative  design  approach  using  derating. 

Government  Industry  Data  Exchange  Program  (GIDEP)  -  is  a  cooperative 
activity  between  Government  and  industry  that  reduces  or  eliminates  expenditures  of 
resources  by  sharing  technical  information  essential  during  research,  design, 
development,  production  and  operational  phases  of  the  life  cycle  of  systems, 
facilities  and  equipment.  Proper  utilization  of  GIDEP  data  can  improve  the 
reliability  of  systems  and  components  during  the  acquisition  and  logistics  phases  of 
the  life  cycle. 

Prohibited  Parts  List  -  is  a  listing  of  parts  deemed  unacceptable  by  the  PMB  for 
use  in  a  company’s  products  because  of  cost,  quality,  safety,  etc. 
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Failure  Mode,  Effects  and  Criticality  Analysis  (FMECA) — or  simply  FMEA — 
identifies  potential  design  weaknesses  through  systematic  analysis  of  the  probable 
ways  (failure  modes)  that  a  component  or  equipment  could  fail.  It  is  a  disciplined 
technique  that  focuses  the  development  of  products  and  processes  on  prioritized 
actions  to  reduce  the  risk  of  product  field  failures,  and  it  documents  those  actions  and 
the  review  process. 

Fault  Tree  Analysis  -  is  a  deductive,  top-down  method  of  analyzing  system  design 
and  performance.  It  involves  specifying  a  top  event  to  analyze  (such  as  a  fire), 
followed  by  identifying  all  of  the  associated  elements  in  the  system  that  could  cause 
that  top  event  to  occur. 

Part  and  Material  Selection  List  -  is  a  part  and  material  listing  preferred  for  use  in 
equipment  design,  which  often  contains  descriptions,  attributes,  or  application 
information. 

Product  Change  Control  -  consists  of  process  and  procedures  that  manage  how 
changes  are  incorporated  into  a  product.  Product  Change  Control  is  a  part  of  overall 
configuration  management  and  uses  review  and  release  processes  to  enforce 
compliance  with  enterprise  change  policies. 

Finite  Element  Analysis  -  is  the  method  of  simulating  a  physical  part  or  assembly’s 
behavior  by  building  a  model  of  the  geometry,  dividing  it  up  into  connected 
elements,  applying  loads  and  constraints,  and  then  calculating  deflections  and 
stresses. 

Failure  Reporting  and  Corrective  Action  System  (FRACAS)  -  is  defined  as  a 
closed-loop  process  for  identifying  and  tracking  root  failure  causes  and  subsequently 
detennining,  implementing  and  verifying  an  effective  corrective  action  to  eliminate 
their  reoccurrence.  The  FRACAS  accumulates  failure,  analysis  and  corrective  action 
infonnation  to  assess  progress  in  eliminating  hardware,  software  and  process-related 
failure  modes  and  mechanisms. 

Circuit  simulation  tools  -  simulates  and  verifies  a  design  for  logic  and  testability 
functions,  timing  behavior,  and  accuracy  to  other  representations  of  the  design 
(behavioral-level,  discrete  breadboard-level,  etc.),  and  performance  in  the  target 
system. 

Design  Verification  Test  (DVT)  -  involves  detailed  tests  to  verify  confonnity  to  all 
applicable  interface  standards,  as  well  as  all  vendor-specific  specifications  and 
diagnostic  commands.  Performed  to  validate  designs,  DVT  is  extremely  rigorous 
and  often  subjects  the  product  to  heat/cool,  electrostatic,  and  EMI/RFI  testing. 

Table  3.  Design  Tools  [From  Ref.  28] 


The  purpose  of  the  second  question  was  to  discover  the  methods  and  techniques 
used  to  determine  parts,  materials,  and  processes  compliance.  Table  4  provides  a  list  of 
the  responses. 
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Receiving  Inspection 

Qualification  Testing 

Environmental  Stress  Screening 

Reliability  Growth  Testing 

Process  Controls 

Reliability  Predictions 

Prohibited  Parts  List 

Evaluation  of  Contractor  PM&P  Program 

Data  Review 

Surveys  and  Audits 

Destruct  Physical  Analysis  (DPA) 

Communication 

Table  4.  Techniques  used  to  detennine  compliance. 


The  third  question  dealt  with  the  methods  and  techniques  that  succeeded.  This 
section  groups  the  responses  into  the  following  categories:  (a)  Testing,  (b) 
Communications,  and  (c)  Data  analysis.  These  categories  were  based  upon  how  the 
interviewees  addressed  the  topic.  Excerpts  from  the  interviews  are  provided  below. 

a.  Testing 

The  following  interview  excerpts  deal  with  the  topic  of  testing. 

When  in  doubt,  test  the  parts.  Example:  on  Program  XXXX,  the  supplier 
qualification  data  for  a  SCR-Diode  module  showed  the  piece  part 
qualification  had  been  done  on  a  sample  size  of  only  ten  pieces,  and 
Autoclave  had  not  been  done  at  all.  We  did  a  full  qualification  test  per  the 
requirements,  with  sample  size  of  50  for  all  tests  except  life  test,  which 
was  done  on  80  parts.  Ten  percent  of  the  modules  failed  temperature 
cycling  and  autoclave  due  to  what  appears  at  this  time  to  be  an  inherent 
design  problem.  The  sample  size  of  ten  pieces  in  the  supplier  qualification 
in  only  one  of  the  tests  failing  was  not  adequate  to  identify  this  problem. 

Plastic  encapsulated  microcircuit  qualification  testing  and  unit 
qualification  testing  [are  important]  because  [they]  screen  out  weak 
suppliers. 

[We  performed]  plastic  encapsulated  microcircuit  sequential  testing;  this 
was  example  of  [an]  independent  test  that  allowed  programs  to  separate 
the  inconsistent  vendors  from  the  consistent  ones. 

b.  Communications 

The  following  excerpts  discuss  communication. 

Require  a  PM&P  Program  that  clearly  outlines  the  procedures  necessary  to 
have  in  place  to  ensure  the  use  of  quality  components  that  will  meet  the 
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overall  system  performance  requirements.  There  needs  to  be  an  expert  in 
this  field  as  part  of  the  team  that  develops  the  plan  and  should  be  involved 
with  any  modifications  to  the  plan.  There  is  a  company  that  has  recently 
developed  a  plan  that  covers  all  three  services  and  was  put  together  with 
input  from  representatives  of  all  three  services.  It  is  up  to  each  program  to 
ensure  the  plan  is  being  implemented  properly. 

We  only  get  data  when  we  fail.  It’s  difficult  to  tell  what  works;  we  have 
tracking  of  what  does  not  work.  Most  times,  the  problem  is  the  contractor 
changing  things  and  not  telling  the  prime.  Adherence  to  processes  and 
communications  is  the  key. 

Supplier  Surveys  and  Audits  are  especially  successful.  If  nothing  else, 
they  give  the  Program  Office  and  contractor  personnel  a  chance  to  meet 
face-to-face  to  discuss  issues  and  problems. 

We  are  also  having  success  with  conducting  supplier-management  surveys 
and  audits.  It’s  been  very  successful  at  getting  insight  into  what  the 
contractor  does,  especially  for  keeping  track  of  process  changes. 

c.  Data  Analysis 

The  following  excerpts  have  to  do  with  data  analysis. 

IPT  working  groups  are  needed  to  analyze  parts  and  materials,  methodically 
reviewing  parts  against  the  application  requirements. 

You  basically  have  to  know  if  the  part  or  material  is  right  for  the 
application  and  if  it  fits  the  mission  profile. 

Proactive  Integrated  Product  Teams  (IPTs)  are  one  of  the  best  vehicles  to 
ensure  product  performance,  reliability  and  repeatability.  This  method 
ensures  all  parties  are  addressing  these  issues  and  are  staying  connected. 
However,  this  only  works  when  all  functionals  participate  and  coordinate 
their  efforts. 

To  ensure  reliability,  you  will  have  to  get  down  to  the  supplier  and  get  into 
his  processes  to  ensure  piece  part  reliability. 


The  fourth  question  dealt  with  the  methodologies  and  techniques  that  have  not 
succeeded.  The  responses  to  this  question  were  interesting  because  most  of  the  answers 
had  nothing  to  do  with  tools  and  techniques.  Instead,  most  of  the  responses  were  about 
communication  and  the  lack  thereof.  This  section  groups  the  responses  into  the  following 
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categories:  (1)  Communications  and  (2)  Design  Process.  Excerpts  from  the  interviews 
are  listed  below. 


d.  Communications 

The  following  excerpts  regard  communication. 

That’s  tough  because  a  lot  here  can  depend  on  how  you  approach  you 
counterpart.  An  aggressive  approach  usually  yields  a  level  of  distrust  and 
standoffness. 

Contractor  Management  of  their  own  PM&P  without  Government 
oversight  [is  needed]. 

We  get  into  trouble  when  we  buy  a  “black  box”  and  don’t  pay  attention  to 
what  PM&P  is  in  it.  An  example  is  the  use  of  pure  tin.  For  most  part 
types  it’s  not  allowed  due  to  tin  whisker  growth.  On  one  program,  the 
subcontractor  had  tin  in  his  black  box,  and  he  did  not  inform  the  prime 
contractor.  The  prime  also  did  not  verify  the  materials  in  the  black  box. 
The  box  failed  and  caused  the  failure  of  a  high-profile,  high-cost  missile 
test. 

[There  is  a]  lack  of  concurrent  engineering  (the  IPPD  process  is  not 
always  rigorously  applied). 

[We]  rely  on  contractors  to  provide  a  system  that  meets  a  perfonnance 
requirement  without  insight. 

I  know  of  no  problems  with  the  tools  or  techniques.  There  was  a  problem 
with  the  subcontractors.  They  were  making  changes  to  processes  without 
telling  the  prime  contractor. 


e.  Design  Process 

The  following  responses  address  the  design  process. 

Destructive  Physical  Analysis  (DPA),  when  not  accompanied  by  a 
comprehensive  PoF  model,  may  not  be  adequate  ...  to  identify  potential 
problems  in  parts. 

The  most  important  thing  is  paying  attention  to  detail. 

The  most  prominent  practice  that  comes  to  mind  is  when  the  contractor 
uses  the  “design  as  you  go”  methodology.  The  design  NEVER  gets 
locked  down,  and  engineers  are  standing  behind  the  operators  changing 
layout  and  components  as  the  CCAs  are  fabricated.  This  often  results  in 
shorts,  opens,  and  numerous  jumper  wires  and  wrong  parts. 
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Not  using  the  design  techniques  I  mentioned  before. 


Contamination  of  plastic  encapsulated  microcircuits  that  will  cause  a  latent 
failure,  intermittent  problem,  or  system  failure.  The  soldering  process  (part  to 
board,  usually  from  a  poor-cleaning  process)  usually  causes  this. 

D.  ACQUISITION  REFORM 

The  second  series  of  interview  questions  focused  on  the  effects  of  acquisition 
reform  on  PM&P  Programs  and  how  PM&P  Program  risks  increased  or  decreased  with 
acquisition  reform.  The  first  question  inquired  how  a  PM&P  Program  was  managed  prior 
to  acquisition  reform.  There  was  almost  complete  agreement  in  the  responses  to  this 
question.  All  of  the  subjects  responded  that  strict  control  was  established  through 
military  specifications  and  standards  that  were  contractually  required.  Below  are  the 
responses. 

MIL-STD-965  was  used.  Parts  needed  to  be  selected  off  a  list,  and  if  it 
was  not  on  the  list,  a  “Non-standard  Part  Request”  was  submitted  to  the 
Government.  Also,  there  was  a  full  level-III  technical  data  package,  and 
all  the  reliability  tasks  of  MIL-STD-785  were  required. 

Use  of  MIL-STD-965.  Use  of  military  specification  parts/established 
reliability  parts  was  the  first  order  of  selection.  Standardization  was 
emphasized. 

MIL-STD-965.  Part  Engineers  would  review  the  final  parts  selected,  for 
compliance  to  standard  requirements  at  the  end  of  the  line. 

Control,  Control,  Control. 

Contractors  were  required  to  adhere  to  military  specifications  and 
standards  as  required  in  their  contracts. 

Design  guidance  called  MIL-STD-1 1991  and  MIL-STD-454  were  used.  It 
specified  what  PM&P  to  use  and  what  not  to  use.  MIL-STD-965  was  used 
for  standardization;  it  also  required  the  use  of  military  specification  parts. 

It  did  not  allow  commercial  parts.  There  were  a  lot  of  data  requirements. 

Data  would  have  to  be  sent  in  to  the  Government  for  review  and  approval 
as  official  data  submittals. 

Control  of  a  level-III  TDP  allowed  for  more  Government  control/insight 
into  the  parts  used. 
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MIL-STD-965  was  required. 


[The]  MIL-STD-965  requirement  was  invoked  in  the  contract.  This 
required  the  contractor  to  submit  all  parts,  and  some  materials,  to  be  used 
in  design,  (except  those  parts  contained  in  the  Government-furnished 
baselines  or  those  previously  approved  by  contract),  for  approval.  The 
problem  for  the  contractor  was  the  time  it  took  to  receive  a  part  approval, 
which  included  subcontractor  part  submission  to  the  prime  contractor  and 
then  prime  contractor  submission  to  the  Government  (not  done 
electronically). 

We  used  two  documents,  MIL-STD-1546  and  MIL-STD-1547.  MIL- 1546 
was  like  MIL-STD-965,  but  more  stringent.  MIL-STD-1547  was  more 
technical;  it  specified  the  exact  screens,  tests,  PDA,  lot  size,  etc.  These 
two  documents  told  contractors  how  to  manage  the  PM&P  Program. 
Contractors  were  required  to  have  a  PM&P  Program,  and  they  had  to  have 
a  PM&P  Control  Board. 

All  the  typical  quality  and  reliability  Military  Specifications  and  Standards 
were  levied  in  the  contract.  The  contractor  was  required  to  adhere  to  the 
military  specifications  and  standards  as  required  in  the  contract.  This 
included  requirements  such  as  MIL-E-11991,  MIL-STD-965,  MIL-STD- 
785,  MIL-STD-1629,  MIL-HDBK-189,  and  MIL-HDBK-78 1 . 


The  second  question  tried  to  ascertain  how  acquisition  reform  changed  the  way 
the  PM&P  Process  is  managed.  All  of  the  responses  reflected  the  fact  that  military 
specifications  and  standards  could  no  longer  be  required.  The  responses  are  shown  below. 


It  shifted  from  an  imposition  of  standards  to  a  process  whereby  the 
contractors  describe  their  processes  and  procedures.  If  these  processes 
and  procedures  do  not  meet  the  goals  of  the  PM&P  Program,  the 
differences  are  negotiated  to  resolution. 

[We]  cannot  use  MIL-STD-1 1991  anymore.  Contractors  now  have  to 
each  create  their  own  PM&P  design  guidance.  Contractors  still  have  to 
get  data  to  make  decisions  and  some  still  submit  data,  but  it  is  on  a 
voluntary  basis  now.  If  a  teaming  arrangement  is  in  place,  most  data  is 
(sic)  obtained  informally,  but  sometimes  travel  is  required  to  go  look  at  it 
because  they  won’t  send  it  out. 

Parts  and  materials  engineers  must  become  more  knowledgeable  and  must 
be  multi-disciplined.  One  of  the  most  important  aspects  of  parts 
engineering  now  is  to  be  a  part  of  the  early  component-selection  process. 
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Parts  Engineering  has  become  an  up-front  activity,  where  in  the  past, 
under  MIL-STD-965,  it  had  become  an  end-of-the-line  activity. 

With  acquisition  reform,  the  Government  can  somewhat  control  the 
process  in  which  parts  and  materials  are  chosen  and  tested,  but  there  is  not 
the  control  down  to  the  piece  part  level  that  was  there  before. 

Less  documentation  required,  screening  and  qualification  levels  lowered 
or  non-existent,  allowance  of  plastic  encapsulated  microcircuits  in  military 
systems,  significant  increase  in  parts  obsolescence,  and  dependence  on 
product  change  notices. 

Too  liberal.  The  response  was  to  throw  away  everything  we  knew  and 
proceed  blindly.  The  move  to  acquisition  reform  was  not  well  thought 
out,  and  companies  reacted  blindly  to  strip  parts  support  from  design 
activities 

It  changed  from  a  MIL-STD-965  requirement  to  a  performance-based 
statement  of  work  type  of  requirement  (examples  contained  in  MIL- 
HDBK-5 12).  The  contractor  would  respond  to  the  SOW  requirement  with 
his  internal  part  management  process.  NAVAIR  would  review  the  process 
to  insure  that  all  of  the  standardization,  reliability,  and  supportability 
issues  were  addressed.  To  ensure  compliance,  NAVAIR  would  have  to 
maintain  insight  through  access  to  the  contractor’s  part  database. 

from  a  contractual  standpoint,  there  were  no  more  “how  to”  requirements; 
neither  were  there  any  more  technical  process  requirements.  The 
underlying  philosophy  is  that  the  contractors  have  their  own  PM&P 
Program.  There  is  some  up-front  assessment  prior  to  contract  award,  but 
there  is  no  longer  a  standard  process  for  all  programs.  Also,  the  contractor 
has  much  more  flexibility  to  express  part-level  test  requirements. 

Military  specifications  for  PM&P  Program  management  were  no  longer 
imposed.  All  the  plans  and  practices  that  were  established  in  the  military 
specifications  were  gone,  and  now  each  contractor  had  to  manage  and/or 
develop  their  own  PM&P  requirements.  Use  of  commercial  parts  was 
emphasized. 

The  contractor  is  now  given  a  performance  specification  in  which  he  is 
required  to  provide  hardware  that  complies  with  the  perfonnance 
requirements  as  stated  in  the  contract.  The  contractor  is  allowed  to  do 
whatever  is  necessary  to  ensure  the  hardware  meets  the  requirements 
stated  in  the  contract. 
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By  tailoring  contractual  language  to  detail  performance  requirements  in 
such  a  way  that  the  contractor  cannot  claim  that  we  are,  in  fact,  telling  him 
“how  to”  build  our  product. 

The  third  question  tried  to  detennine  if  risks  relative  to  the  PM&P  Program 
increased  or  decreased  with  acquisition  refonn.  There  was  almost  complete  agreement 
that  the  risks  had  increased.  Below  are  the  responses.  The  lone  exception  was  an 
individual  who  worked  on  a  weapon  system  that  was  still  using  military  specification  and 
standards  requirements.  The  program  had  negotiated  an  agreement  with  the  contractor  to 
keep  using  the  requirements,  and  that  subject  stated  that  there  was  no  change  in  risk.  The 
responses  are  listed  below. 


Risks  are  especially  high  in  organizations  where  Parts  Engineers  are  not 
knowledgeable  and  are  not  involved  in  the  component  selection  process. 

Risk  has  increased,  but  it  is  dependent  on  the  Project  and  the  people 
working  in  the  project.  Communication  is  the  key  .... 

Risk  increased  due  to  [the]  answer  above,  but  risk  also  decreased  due  to 
use  of  IPPD  process  (better  coordination  between  disciplines,  more 
tailoring  of  selected  capabilities  to  application  requirements). 

The  risks  increased.  We  relinquished  control  (as  required  by  no  contract 
requirements),  and  the  risks  increased. 

Risk  has  increased  because  no  standard  methodology  of  parts  management 
is  in  place. 

In  my  opinion,  risk  has  increased  significantly.  The  Government  tried  to 
balance  cost  and  resources  and  it  did  not  work.  The  contractor  needed  to 
modify  processes  to  lower  the  bid  price,  so  they  streamlined  the 
specifications  and  standards,  along  with  personnel  and  processes. 

Risks  to  the  Government  increased.  This  was  due  to  the  fact  that  we  only 
knew  what  the  contractor  did,  as  far  as  part  decisions,  after  the  fact,  and  in 
some  cases,  due  to  a  lack  of  personnel,  not  at  all. 

Risk  has  not  increased  or  decreased  at  all  on  the  program  [where]  I  work. 
We  still  use  all  the  military  reliability  requirements. 

Risk  has  increased,  but  it  is  dependent  on  the  Program  Office  and  the 
people  working  there.  Communication  is  the  key;  if  it  is  good,  the  risk  is 
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lower,  [and]  if  it  is  bad,  then  risk  has  increased  due  to  the  lack  of  oversight 
and  inadequate  peer  review. 


Risks  increase  for  contractors  who  are  not  willing  to  negotiate  towards  the 
goals  of  the  PM&P  Program;  the  risks  decrease  for  contractors  who  are 
willing  to  negotiate  in  good  faith  in  order  to  deliver  quality  hardware. 

Definitely  much  more  risk.  It  is  next  to  impossible  to  get  a  contractor  to 
do  anything  outside  of  their  normal  operating  procedures  (which  are 
usually  under  the  auspices  of  “best  commercial  practices”)  without  hard 
contractual  requirements  to  back  them  up. 

E.  COMMERCIAL  INDUSTRY 

The  third  series  of  interview  questions  focused  on  the  methodologies  and 
techniques  that  commercial  industry  uses  to  manage  parts,  materials,  and  processes  and 
what  commercial  industry  techniques,  if  any,  the  DoD  could  adopt.  The  first  question 
asked  about  the  methodologies  and  techniques  commercial  industry  uses  to  manage  parts, 
materials,  and  processes.  These  are  the  responses: 


They  do  not  really  do  anything  differently.  They  have  more  of  the  ability 
to  start  research  or  perform  special  testing  (if  necessary)  on  new  materials. 
A  lot  of  times,  commercial  industry  can  be  more  tolerant  of  failures 
(depending  on  market  impact),  such  as  poor  production  yield. 

I  cannot  comment,  as  I  have  no  experience  with  commercial  OEMs. 

Not  sure. 

They  depend  on  Process  Change  Notices  (PCNs)  from  part  suppliers. 
They  also  do  more  first-article  qualification  at  the  part  level. 

They  are  selective  in  suppliers  they  deal  with  and  they  use  “agreements” 
to  control  proliferation.  They  also  manage  their  suppliers  much  more 
efficiently  than  military. 

If  commercial  industry  has  processes,  they  would  closely  resemble  those 
of  the  Defense  Contractors.  Defense  Contractors  only  developed  their 
processes  because  of  MIL-STD-965. 

Commercial  companies  certainly  do  PM&P  Management,  but  it  is  a  totally 
different  business.  The  make  use  of  strong  incentives  and  high  volume 
production.  The  incentive  is  the  opportunity  for  follow-on  business.  The 
big  difference  between  commercial  and  military  is  that  commercial 
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industry  uses  high  volume  and  [a]  low  number  of  part  types.  The  military 
is  just  the  opposite.  This  gives  the  commercial  industry  a  lot  more 
leverage. 

Some  companies  are  trying  to  develop  their  own  design  guidance.  The 
problem  is  that  it  is  high-order  and  not  detailed.  However,  some  good 
documents  do  exist  for  guidance,  including  the  AIAA  R-100,  SAE  R&M 
100,  AEC  QS9000  documents,  and  the  Stack  International  documents 
(these  document  are  based  on  the  old  military  specification  and  standards). 

In  a  general  sense,  they  each  have  company  standard  processes  that  they 
use.  But  each  company  and  each  division  within  a  large  company  all  use 
their  own  peculiar  process.  Some  don’t  even  manage  PM&P.  Some  still 
use  military  specifications,  but  most  have  abandoned  due  to  cost. 

When  it  is  to  their  financial  advantage,  the  contractors  will  adhere  to  select 
guidelines  contained  in  the  PMP.  It  has  been  my  experience,  however, 
that  they  will  always  do  what  is  the  fastest,  cheapest  and  easiest  process. 

I  will  take  a  pass  on  this  one;  I  don’t  have  insight  into  commercial 
processes. 

The  purpose  of  the  second  question  was  to  see  if  the  respondents  thought  that 
DoD  could  adopt  any  commercial  industry  techniques.  As  might  be  expected  after 
reading  the  responses  to  the  above  question,  the  subjects  did  not  think  that  many 
commercial  techniques  could  be  utilized.  Five  either  did  not  know  of  any  commercial 
techniques  or  could  not  think  of  any  that  DoD  could  use.  Below  are  the  responses  of 
those  who  did  have  an  answer. 


Concurrent  engineering  and  IPPD,  which  we  are  using. 


Contractors  have  the  ability  to  quickly  obtain  expertise  from  experts  when 
they  have  PM&P  issues.  There  are  a  number  of  consulting  firms  that  will 
provide  this  service.  However,  we  have  teamed  with  most  of  our 
contractors  to  create  basic  methodologies  based  upon  a  combination  of  our 
lessons  learned  and  their  lessons  learned  information.  I  don’t  know;  I 
don’t  see  a  lot  of  things  different.  We  do  try  to  utilize  any  industry 
document  that  we  feel  is  applicable  to  missile  systems — e.g.,  AEC,  EIA, 
Stack  international,  etc.  Also  within  other  Government  agencies — for 
example,  the  Department  of  Energy  has  just  forwarded  us  a  preliminary 
design  guide  that  one  of  their  subcontractors  has  developed  for  the  nuclear 
people. 
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In  the  past,  DoD  has  not  been  willing  to  pay  the  upfront  cost  needed  to  set 
a  “commercial”  infrastructure.  Too  many  rules  and  regulations.  It’s  not 
politically  correct.  Until  this  “control”  without  payment  idea  is  changed, 
it  will  be  difficult  to  adopt  the  buyer/seller  relationship,  guaranteed 
procurements,  standardized  products  and  block  upgrades. 

None  that  I'm  aware  of. 

It’s  difficult  to  say.  The  consumer  product  is  very  different  from  what  the 
military  requires.  The  military  requires  very  robust,  long  lasting  parts. 

The  commercial  world  does  not  necessarily  test  parts,  even  though  they 
guarantee  parameters.  The  military  requires  actual  testing  and  not  just 
guarantees.  It’s  also  hard  to  leverage  off  the  commercial  world  because 
they  design  to  a  different  level.  Some  projects  are  doing  “up-screening”  to 
get  parts  that  perfonn  adequately. 

This  all  depends  on  the  DoD  system.  If  the  DoD  is  developing  a  system 
that  is  typical  to  the  environments  and  lifespan  of  commercial  hardware, 
then  we  can  adopt  many  of  the  industry  techniques.  If,  however,  we  are 
developing  a  system  that  is  to  be  deployed  in  [a]  very  harsh  military 
environment  with  fairly  lengthy  life  cycles,  then  we  need  to  be  very 
careful  what  techniques  we  allow.  We  are  currently  allowing  the  use  of 
certain  test  data  from  microcircuit  manufacturers  to  provide  confidence 
that  the  manufacturing  process  used  is  sufficient  to  produce  quality 
components.  In  some  cases,  manufacturers  do  not  provide  sufficient  data 
to  warrant  this,  and  further  tests  are  needed  to  provide  that  confidence. 

F.  MITIGATING  PM&P  RISK 

The  fourth  interview  question  was  about  things  a  Program  Manager  can  do  to 

mitigate  part  and  material  risks.  The  question  asked  the  top  three  things  a  Program 

Manager  can  do  to  mitigate  part,  material,  and  process  risk  and  maximize  reliability  in 

the  design.  The  responses  below  list  the  interviewees'  suggestions. 

1)  Recognize  that  they  need  to  perfonn  qualification  testing  at  the  part  and 
material  level;  2)  require  the  contractor  to  implement  a  comprehensive 
FRACAS  down  to  the  part  level;  3)  monitor  suppliers  for  changes  to 
PM&P  that  would  affect  performance. 

1)  Plan  for  adequate  funding  in  the  reliability  area  and  don’t  take  it. 
Programs  usually  take  money  from  test  and  reliability  for  other  priorities; 

2)  Have  project  office  personnel  participate  in  all  (or  as  many  as  possible) 

IPT  meetings;  3)  emphasize  communication  at  all  levels.  Some  programs 
keep  too  much  information  “close  hold”  and  do  not  have  adequate  peer 
review  with  subject  matter  specialists. 
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1)  Ensure  that  highly  knowledgeable  commercial  component  experts  are 
involved  in  the  early  concept  process  and  the  early  part  selection  process; 

2)  allocate  funding  for  independent  testing  of  piece  parts,  and  let  the 
Component  Engineers  know  that  there  are  no  barriers  to  testing  of  parts 
where  they  feel  it  is  required;  3)  require  that  the  entire  part  review  for  each 
part  be  documented;  perform  QC  audits  of  the  PM&P  review  process  early 
in  the  Design,  and  periodically  throughout  the  design  process. 

1)  Develop  a  comprehensive  PMP  plan  as  early  as  possible  in  the  system 
life  cycle;  2)  maintain  support  from  PMP  “experts”  throughout  the 
program;  3)  make  sure  the  performance  spec  has  adequate  requirements  to 
enforce  the  PMP  program. 

1)  Provide  adequate  budget  for  detailed  part  assessments;  2)  enforce  part 
test  and  qualification  requirements;  3)  use  a  piece  part  level  FRACAS  that 
requires  root  cause  analysis  down  [to]  the  part  level. 

1)  Establish  the  PMP  rules  and  authority  immediately  after  contract  award; 

2)  empower  the  PMP  organization  to  do  its  job  within  the  boundaries 
outline  by  the  first  bullet;  3)  support  the  PMP  activities  within  the 
engineering  design  organization. 

1)  Write  good  contract  requirements;  2)  invoke  a  Parts  Management 
Program  (PMP)  requirement  in  the  contract;  3)  assign  a  PM&P  Program 
IPT  lead  to  insure  that  all  part  technical  issues,  including  "lessons 
learned,"  are  addressed. 

1)  Know  and  understand  your  part  and  material  application;  2)  know  and 
understand  the  inherent  design  of  the  parts  and  materials;  3)  know  your 
part  and  material  suppliers. 

1)  [Have  a  ]  Parts  Management  Plan;  2)  use  and  support  effective  Testing 
Techniques  (part  level,  board  level,  DVT);  3)  use  adequate  design  tools. 

1)  He  needs  a  PM&P  Plan  that  includes  testing  at  the  part  level;  2)  need  to 
perform  construction  analysis  on  parts  to  verify  changes;  3)  need  a  closed 
loop  FRACAS  to  avoid  long-term  reliability  problems. 

1)  Allow  early  part  expert  review  of  selection  of  parts  and  materials;  2) 
fund  qualification  testing  of  parts  and  materials;  3)  fund  root  cause  and 
corrective  action  of  part  failures  early  in  design  and  during  production. 
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G.  INTERFACE  WITH  LOGISTICS 


The  last  series  of  interview  questions  focused  on  the  type  and  quality  of  input  a 
PM  gets  from  the  LM  on  part,  material,  and  process  issues.  The  first  question  asked  how 
much  and  what  type  of  input  the  PM  gets  from  the  LM  on  part,  material,  and  process 
issues.  There  was  overwhelming  consensus  that  there  is  little  support  from  the  LM. 
Below  are  the  responses. 

Logistics  does  not  do  much  up  front. 

I  work  several  programs,  and  I  only  know  of  one  program  that  gets  input 
from  the  LM;  otherwise  it  is  non-existent.  The  type  of  input  usually 
revolves  around  supportability  issues.  Actually,  what  I  am  trying  to  say  is 
I  only  have  one  program  where  the  logistics  people  work  closely  with 
other  engineers,  such  as  this  office,  to  provide  the  best  advice  to  the  PM. 
Usually  logistics  works  largely  independent  of  everyone  else,  it  appears. 

All  programs  get  input  from  logistics,  but  on  other  programs  they  don’t 
always  seem  to  work  with  other  organizations. 

I  cannot  comment  on  this,  as  I  have  not  interfaced  with  Logistics. 

Very  little  that  I  have  seen. 

They  provide  very  little  input,  only  minimal  input  of  field  failure  data  and 
standardization. 

None  that  I  am  aware  of. 

These  questions  are  difficult  to  answer  because  generally  there  is  not  any 
input  from  the  Logistics  Manager. 

Don’t  know,  our  organization  does  not  get  much  input  from  logistics. 

Logistics  is  not  really  involved. 

The  space  world  does  not  do  sustainment;  once  it's  launched,  that’s  it. 

Usually  nothing. 


The  second  question  asked  how  the  level  of  input  could  be  improved,  and  why. 
Again  there  was  an  overwhelming  consensus;  this  time,  most  respondents  thought  the 
level  of  support  from  the  LM  needs  improvement.  Below  are  the  responses. 
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Yes,  they  could  provide  the  logistics  impacts  of  the  early  design. 

Yes,  it  definitely  could  because  when  the  LM  understands  the  importance 
of  reliability  and  when  he  provides  good  support  and  works  with  other 
functional  support  people,  the  PM  gets  the  better  recommendations. 

Yes,  it  would  help  with  supportability  type  issues. 

Yes,  he  needs  to  be  involved.  He  is  the  one  that  has  all  the  lessons  learned 
from  the  field.  Education  of  the  logistics  people  is  a  big  issue. 

Yes,  it  could  be  improved,  but  it  might  be  hard  to  get  logistics  involved. 
In  the  past,  they  have  offered  support  on  issues  such  as  transportability. 

I  cannot  comment  on  this,  as  I  have  not  interfaced  with  Logistics. 

Needs  to  be  improved. 

Yes,  they  could  provide  better  feedback  at  the  part  and  material  level  from 
the  field  and  depots. 

Yes,  he  needs  to  be  a  big  part  of  this  because  obsolescence  is  such  a  big 
issue  with  short-life-cycle  items  like  computer  parts. 

I  am  not  sure.  Perhaps  they  could  provide  input  into  specification 
requirements. 

It  is  a  matter  of  philosophy  of  the  program.  I  believe  with  the  level  of 
integration  and  pin  count  density  has  lead  the  logisticians  to  stop  at  the 
LRU/CCA  level  (since  a  depot  probably  should  not  be  perfonning  piece 
part  removal  and  replacement). 


The  third  and  fourth  questions  asked  the  respondents  to  say  if  the  input  from  the 
LM  is  useful  and  to  give  examples  (good  or  bad)  of  that  input.  The  subjects  typically  did 
not  answer  these  two  questions  because  of  the  lack  of  contribution  from  the  LM.  In  the 
rare  times  that  the  LM  offered  input,  some  thought  it  useful  and  some  did  not.  Below  are 
the  responses  regarding  input  from  the  LM. 


Not  really.  The  logistics  manager  appears  to  be  lacking  the  technical  skills 
needed  to  impact  the  early  design. 

Yes,  very  useful.  Like  I  said  before,  I  think  it  is  most  useful  when  the 
logistics  works  closely  with  reliability,  engineering  and  other  groups  and 
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the  projects.  The  LM  offered  input  on  how  the  equipment  would  be  used 
by  the  soldier  in  the  field.  After  discussions  with  product  assurance 
people  and  other  engineering,  it  caused  a  design  change  that  would  not 
have  otherwise  been  made. 

All  feedback  from  Logistics  is  useful  since  it  allows  the  PM  to  alter  the 
design,  if  necessary,  to  account  for  issues  seen  in  the  field. 

Input  is  only  minimally  useful  at  the  LRU  or  subsystem  level. 

At  my  level  of  insight,  typically  no.  When  it  comes  to  obsolescence,  the 
length  of  time  it  takes  the  logistics  community  to  determine  what  assets 
they  are  in  need  of,  the  window  of  opportunity  has  closed. 

H.  CONCLUSION 

Eleven  SMEs  in  the  parts,  materials,  and  processes  functional  area  described  their 
experiences  working  on  weapon  systems.  The  SMEs  represented  programs  from  the 
Anny,  Navy,  and  the  Air  Force.  Each  person  discussed  characteristics  of  the  program  or 
programs  he  works  on.  The  responses  were  indicative  of  their  personal  experience  in 
working  on  PM&P  Programs.  The  next  chapter  contains  an  analysis  of  these  data  and  an 
attempt  to  develop  a  set  of  heuristics  to  assist  future  Program  and  Logistics  Managers. 
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IV.  ANALYSIS 


A.  INTRODUCTION 

This  chapter  analyzes  the  data  presented  in  the  previous  chapter.  The  analysis  is 
based  on  the  interview  responses  of  participating  subjects  and  is  structured  around  the 
same  topics  as  in  Chapter  III.  The  analysis  consists  primarily  of  comparing  the  survey 
responses  to  one  another,  searching  for  common  issues  and  identifying  trends.  The 
researcher  uses  this  analysis  to  identify  strategies  and  tactics  for  optimizing  the  use  of 
highly  reliable  parts,  materials,  and  processes  in  order  to  maximize  system-inherent 
reliability  in  the  life  cycle. 

B.  CLASSIFYING  INTERVIEW  DATA 

Analysis  of  the  interview  responses  was  conducted  according  to  the  classification 
scheme  presented  in  Chapter  III.  The  interview  responses  were  organized  in  relation  to 
the  secondary  research  questions,  and  a  comparison  was  made  to  yield  trends  and 
commonality.  In  general,  the  secondary  research  questions  pertain  to  five  categories:  1) 
the  technology,  tools,  and  techniques  available  to  ensure  the  proper  selection  of  reliable 
parts  and  materials;  2)  the  way  in  which  acquisition  reform  has  changed  how  a  PM&P 
Program  is  managed;  3)  the  extent  to  which  commercial  industry  differs  in  its  approach  to 
part  and  material  selection;  4)  how  a  Program  Manager  can  mitigate  part  and  material 
risks;  and  5)  how  a  Logistics  Manager  interfaces  with  the  PM&P  Program. 

C.  ANALYSIS  OF  KEY  ISSUES 

1.  Technology,  Tools,  And  Techniques 

What  enablers  allow  the  selection  and  verification  of  highly  reliable  parts, 
materials,  and  processes?  From  the  data  gathered,  there  appear  to  be  numerous  tools  and 
techniques  that  can  be  used  to  ensure  that  the  design  of  a  weapon  system  is  maximized 
for  reliability. 

Table  3  of  Chapter  III  listed  and  defined  the  technologies,  tools,  and  techniques 
that  were  mentioned  in  the  interviews.  Table  5  below  lists  the  six  most  frequently 
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mentioned  responses  (mentioned  by  at  least  four  of  the  subjects).  Definitions  of  these 
tools  are  listed  in  Table  3,  Chapter  III. 


Top  Six  Design  Tools  That  Are  Used  To  Ensure  That 
Reliability  Is  “Built  In”  Early  In  The  Design  Of  A 
Weapons  System 

Reliability  Prediction  Analysis 
Circuit  Tolerance  Analysis 
Sneak  Circuit  Analysis 
Physics  of  Failure  Tools 
Part  Derating 

Government  Industry  Data  Exchange  Program 
Table  5.  Top  Six  Design  Tools 

Proven  and/or  disciplined  design  approaches  are  a  critical  factor  in  the  reliability 
success  of  a  product.  The  tools  and  techniques  in  Table  5  were  identified  in  the 
interviews  as  most  utilized  by  programs  in  weapon  system  design  (part  and  material 
selection  and  control). 

The  four  most  frequently  mentioned  methodologies  and  techniques  used  to 
determine  compliance  are  listed  in  Table  6.  Interestingly,  all  but  two  of  the  interview 
subjects  stated  that  some  sort  of  testing  was  necessary  to  verify  compliance.  One  even 
responded  with,  “Testing,  testing,  testing,  (i.e.,  all  types:  board-level,  part-level, 
independent  testing,  DVT,  and  special  testing),  all  types  and  levels  are  important.”  The 
responses  were  a  strong  indicator  that  testing  at  all  levels  is  critical  to  verify  part  and 
material  compliance.  All  of  the  interviewees  mentioned  that  either  testing  or  some  type 
of  data  review  is  necessary  ensure  compliance.  Even  the  establishment  of  a  PM&P 
Program  was  secondary  to  testing. 
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Top  Four  Methodologies  And  Techniques  Used  To 
Determine  Part,  Material,  And  Processes  Compliance 

Testing  (i.e.,  Environmental  Stress  Screening, 
Qualification  Testing,  Design  Verification  Testing, 
Independent  Testing) 

Evaluation  of  Contractor  PM&P  Program 


Data  Analysis  and  Review 


Surveys  and  Audits 


Table  6.  Top  Four  Verification  Methods 

The  answers  to  the  question  about  successful  methodologies  and  techniques 
provided  the  common  themes  listed  in  Table  7.  Adequate  communication  was  the  answer 
mentioned  by  most  people.  Everyone  responding  also  said  something  about  perfonning 
some  type  of  testing  or  data  review.  There  already  is  a  recurring  theme  that  testing  needs 
to  be  part  of  a  PM&P  Program. 


Successful  Methodologies  And  Techniques 

Communication  through  IPTs 
Part  and  Material  Testing. 

Surveys  and  Audits 

Table  7.  Successful  Methodologies  and  Techniques 

The  subject  of  methodologies  and  techniques  that  have  not  succeeded  did  not 
yield  the  type  of  answers  expected.  This  was  intended  to  be  an  assessment  of  the  tools 
and  techniques  that  were  identified  in  Table  3  of  Chapter  III.  Only  one  of  the  answers 
had  something  to  do  with  tools  and  techniques;  most  of  the  responses  were  about 
communication  and  the  lack  thereof.  Though  unexpected,  the  answers  were, 
nevertheless,  consistent  with  the  answers  to  the  previous  question.  One  interview  subject 
stated  that  “the  lack  of  concurrent  engineering”  stood  out  in  his  mind  as  the  biggest  issue. 
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Another  response  was  that  “communications  is  vital.”  Someone  else  said,  “It  depends  on 
how  you  approach  you  counterpart.  An  aggressive  approach  yields  a  level  of  distrust.”  It 
was  evident  that  all  the  respondents  believe  communication  to  be  one  of  the  more 
important  aspects  of  a  successful  PM&P  Program.  In  a  separate  discussion,  one 
individual  remarked: 

I  do  not  believe  there  are  any  tools  you  can  use  to  build  in  reliability. 

Select  the  right  part  from  the  right  supplier  based  upon  all  the  available 

data  you  have  ...  to  ensure  reliability,  you  will  have  to  get  down  to  the 

supplier  and  get  into  his  processes  to  ensure  piece  part  reliability. 

This  and  the  other  statements  strongly  imply  that  communication  between  all  the 
parties  involved  is  the  most  critical  factor. 

2.  Acquisition  Reform 

This  section  analyzes  the  effects  of  acquisition  reform  on  PM&P  Programs. 
There  was  complete  consensus  about  the  way  PM&P  Programs  were  managed  prior  to 
acquisition  reform.  Consistent  with  the  literature  review,  the  common  theme  was  that 
strict  Government  control  was  established  through  military  specifications  and  standards. 
MIL-STD-965  was  the  prevailing  document  that  compelled  the  use  of  a  standard  process. 
Along  with  the  requirements  of  MIL-STD-965,  one  person  said  that  “there  were  a  lot  of 
data  requirements,  and  data  would  have  to  be  sent  to  the  Government  for  review  and 
approval.” 

For  changes  after  acquisition  reform,  there  was  universal  agreement  that  military 
specifications  and  standards  could  no  longer  be  used.  This  was  also  as  expected  from  the 
literature  review.  Because  of  the  change  to  perfonnance-based  contracting,  the  level  of 
control  enabled  by  military  specifications  and  standards  was  no  longer  there.  The  focus 
shifted  from  controlling  the  design  of  the  product  to  having  insight  into  the  contractor’s 
processes.  The  answers  also  indicated  an  increased  dependence  on  the  ability  to 
communicate.  No  longer  could  a  program  sit  back  and  wait  for  data  to  be  delivered.  If  a 
program  wanted  insight  at  the  part  and  material  level,  personnel  had  to  be  proactive  and 
get  the  infonnation  by  participating  in  the  IPT  process.  The  result  of  not  being  able  to 
use  military  specifications  and  standards  can  be  grouped  into  three  categories:  1)  there  is 
no  longer  one  uniform  process  with  strict  Government  control;  2)  there  is  an  inability  to 
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utilize  the  lessons-learned  contained  in  the  military  specifications  and  standards;  and  3) 
requirements  had  to  be  specified  in  perfonnance-based  terminology. 

On  the  question  of  post-acquisition  reform  risk,  there  was  an  overwhelming 
consensus  that  acquisition  reform  had  increased  risk.  All  of  the  survey  respondents  were 
in  complete  agreement,  except  for  one  individual,  who  said  that  risk  had  not  changed. 
His  program  had  negotiated  with  the  contractor  to  maintain  the  requirements  that  were  in 
the  cancelled  military  specification.  Even  without  taking  this  into  account,  its  obvious 
that  acquisition  reform  is  not  held  in  high  regard  in  the  parts,  materials,  and  processes 
arena.  The  most  interesting  part  was  that  even  the  contractors  who  were  interviewed  also 
agreed  that  acquisition  reform  had  increased  risk.  This  is  interesting  because  one  aspect 
of  acquisition  refonn  was  to  provide  contractors  relief  from  Government-unique  terms 
and  conditions.  [Ref.  29]  One  of  the  contractors  stated,  “Too  liberal.  Response  was  to 
throw  away  everything  we  knew  and  proceed  blindly.  The  move  to  acquisition  reform 
was  not  well  thought  out,  and  companies  reacted  blindly.”  Another  contractor  said, 
“Design  Engineers  have  other  priorities  which  often  conflict  with  the  selection  of  highly 
reliable  parts.” 

Through  all  the  interviews,  one  positive  theme  emerged  from  the  discussions  of 
acquisition  refonn.  That  was  the  use  of  IPTs.  Concurrent  engineering  and  the  use  of 
IPTs  were  mentioned  as  a  method  that  has  reduced  some  of  the  risks  that  were  created.  It 
did  this  in  two  ways:  1)  by  increasing  the  flow  of  communication,  and  2)  by  involving 
parts  and  materials  engineers  in  the  earlier  stages  of  the  design. 

3.  Commercial  Industry 

This  section  analyzes  the  techniques  that  commercial  industry  uses  to  manage 
parts,  materials,  and  processes  and  which,  if  any,  DoD  could  adopt.  The  responses  here 
varied  greatly.  A  number  of  the  interview  subjects  stated  that  they  were  unfamiliar  with 
commercial  industry's  practices.  It  is  apparent  that  commercial  industry  has  not,  as 
envisioned,  stepped  up  and  developed  “best  practices”  or  initiatives  to  replace  part  and 
material  military  standards  and  specifications.  The  common  theme  among  the 
individuals  who  did  answer  this  question  was  that  commercial  industry  does  not  really  do 
anything  differently  from  the  military.  In  fact,  it  appears  that  commercial  industry 
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leverages  off  the  DoD  for  part  and  material  requirements  and  testing.  When  something  is 
done  differently,  such  as  special  testing,  it  is  because  commercial  industry  operates  in  a 
more  flexible  environment.  It  is  also  apparent  that  commercial  industry  is  a  different 
type  of  business.  Because  commercial  businesses  are  a  bigger  part  of  the  market,  they 
have  more  leverage  with  suppliers.  Also,  depending  on  the  market  impact,  they  also  can 
be  more  tolerant  of  failures.  Due  to  safety  and  readiness  reasons,  the  military  is  typically 
not  as  tolerant  of  failures. 

As  might  be  expected  from  the  above  responses,  the  subjects  did  not  think  that  the 
military  could  use  very  many  commercial  techniques.  In  fact,  the  first  common  theme 
that  emerged  was  that  the  military  already  leverages  commercial  practices,  such  as 
concurrent  engineering  and  IPTs.  The  other  theme  is  that  the  DoD  does  not  want  to  use 
some  commercial  practices  because  of  the  differences  between  the  two  markets.  The 
military  application  environment  is  much  more  severe  than  the  typical  commercial 
application.  One  person  said,  “If  we  are  developing  a  system  that  is  to  be  deployed  in  [a] 
very  harsh  military  environment  with  fairly  lengthy  life  cycles,  then  we  need  to  be  very 
careful  what  techniques  we  allow.”  Overall,  there  seemed  to  be  agreement  that  the  DoD 
was  already  leveraging  off  commercial  practices  where  it  was  practical. 

4.  Mitigating  PM&P  Risk 

This  section  analyzes  the  tools  and  practices  that  a  PM  can  use  to  mitigate  part, 
material,  and  process  risks.  The  purpose  of  this  question  was  to  find  synergy  among  the 
experts  with  regard  to  managing  a  highly  reliable  PM&P  Program.  Table  8  provides  a 
summary  of  the  most  frequent  responses. 
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The  Top  Things  A  Program  Manager  Can  Do  To  Mitigate  PM&P 
Risk  And  Maximize  Reliability  In  The  Design 

Provide  adequate  funding  for  a  PM&P  Program  that  includes  testing  at 
the  part  and  material  level. 

Develop  a  comprehensive  PM&P  Program  as  early  as  possible  in  the 
system  life  cycle. 

Ensure  that  adequate  parts,  materials,  and  process  requirements  are 
placed  in  contracts. 

Involve  PM&P  experts  early  in  the  process. 

Implement  a  FRACAS  down  to  part  and  material  level. 

Monitor  and  emphasize  communication  with  suppliers. 

Utilize  IPTs,  including  a  PM&P  IPT. 

Table  8.  The  Top  Things  A  Program  Manager  Can  Do  To  Mitigate  PM&P  Risk 

It  was  interesting  that  none  of  the  responses  had  anything  to  do  with  the  tools  and 
techniques  mentioned  earlier.  Three  main  themes  developed:  1)  establish  a  PM&P 
Program  early  in  the  system  life  cycle  that  includes  testing  at  the  part  and  material  level; 
2)  institute  strong  management  support  (funding  and  enforcement);  and  3)  make 
communication  between  all  parties  a  part  of  the  process. 

5.  Interface  with  Logistics 

This  section  analyzes  how  much  and  what  kind  of  input  the  PM  gets  from  the 
Logistics  Manager  in  a  PM&P  Program.  The  purpose  of  this  series  of  questions  was 
twofold;  it  was  an  attempt  to  get  an  indication  of  the  level  of  support  being  provided  and 
then  to  find  out  how  this  support  could  be  improved.  From  the  list  of  responses  in 
Chapter  III,  it  is  readily  apparent  there  is  little  support  from  the  LM  on  part,  material,  and 
process  issues.  Only  one  of  the  interview  subjects  even  knew  of  a  program  where  the 
LM  was  involved 

There  was  also  a  resounding  consensus  that  the  level  of  support  needs  to  be 
improved.  Quite  a  few  interviewees  commented  that  the  LM  has  expertise  in  fielded 
equipment  and  should  share  this  information  with  the  design  team.  Answers  such  as  “the 
LM  understands  the  importance  of  reliability”  and  “they  could  provide  better  feedback  at 
the  part  level,  from  the  field  and  depots”  demonstrate  that  there  is  a  need  for  this 
information. 
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As  far  as  the  usefulness  of  the  input  from  the  LM  is  concerned,  there  was  a  mixed 
reaction.  There  typically  was  no  answer  given  for  these  two  questions  because  of  the 
lack  of  contribution  from  the  LM.  On  the  rare  occasions  that  there  was  input  from  the 
LM,  some  thought  it  useful  and  some  did  not.  The  answers  indicated  that  education  of 
the  logistics  personnel  is  a  key  issue.  One  person  responded  that  “the  logistics  group 
does  not  have  the  expertise  required  to  impact  the  early  design”  and  that  “logistics 
managers  appear  to  be  lacking  the  technical  skills  needed  to  impact  the  early  design.”  All 
indications  are  that  logistics  is  currently  not  a  part  of  the  design  process,  where  they 
could  have  the  most  impact. 

D.  LESSONS  LEARNED 

The  following  lessons  can  be  extracted  from  the  literature  review  and  the 
interview  responses. 

1.  “Design  in”  Reliability  Using  Highly  Reliable  Parts,  Materials,  and 

Processes 

Achieving  reliable,  available  and  maintainable  systems  requires  a  disciplined 
systems  engineering  approach  that  starts  early  on  in  a  program.  Reliability  can  be 
achieved  with  the  help  of  highly  reliable  parts,  materials,  and  processes.  It  is  not 
something  that  can  be  tested  in  later;  it  must  be  part  of  the  design  process  while  its  still 
cost-effective. 

2.  Know  And  Understand  Your  Part  And  Material  Application 

Requirements 

There  needs  to  be  a  clear  determination  of  the  system  performance  requirements, 
including  reliability.  The  system  requirements  will  get  allocated  to  the  lowest  levels  in 
the  design.  The  impacts  of  not  establishing  a  proper  requirement  will  reverberate  down 
to  the  part  and  material  level. 

3.  Test  At  All  Levels  of  the  Design;  Test  Early;  Test  Often 

The  ability  to  design  a  reliable  system  requires  early  and  continuous  testing  from 
the  system  level  down  to  the  parts  and  materials  that  make  up  the  system.  This  testing  is 
necessary  in  order  to  verify  and  monitor  successful  achievement  of  all  the  performance 
requirements,  including  reliability. 
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4.  Know  Your  Part  Suppliers 

Reliability  issues  are  not  always  at  the  system  or  Line  Replaceable  Unit  (LRU) 
level.  Details  are  required  all  the  way  down  to  the  part  and  material  level.  A 
microcircuit  the  size  of  a  penny  can  end  up  causing  a  multi-million-dollar  weapon  system 
to  fail.  Attention  to  the  underlying  manufacturing  design  and  manufacturing  processes  of 
the  contractor,  their  subcontractors,  and  vendors  are  needed  to  ensure  part  and  material 
reliability. 

5.  Know  And  Understand  The  Inherent  Design  Of  The  Parts 

Predicted  perfonnance,  even  at  the  part  and  material  level,  can  be  overstated. 
Apply  a  null  hypothesis  to  these  claims — i.e.,  they  are  untrue  until  proven  otherwise  in 
the  form  of  valid  test  results  with  confidence  in  the  numbers.  Just  because  a  supplier 
claims  something  on  the  data  sheet  does  not  mean  that  the  product  has  been  verified  at 
those  parameters.  Use  available  data  if  it  is  applicable  to  your  system,  but  always  have  a 
contractor  prove  his/her  reliability  claims,  even  at  the  part  and  material  level. 

6.  Communication  is  a  Key  Part  of  a  Successful  PM&P  Program 

Successful  communications  will  ensure  the  best  possible  outcome  of  any  plan  or 
program,  and  this  is  no  different  for  a  PM&P  Program.  This  will  require  communication 
and  coordination  among  customers,  design  engineers,  logisticians,  contractors,  etc.  A 
program  with  an  atmosphere  of  open  dialogue  and  communication  is  well  on  its  way  to 
achieving  its  long-tenn  goals. 

E.  SUMMARY 

The  analysis  in  this  chapter  provides  a  valuable  insight  into  the  issues  that  affect 
the  use  of  highly  reliable  parts,  materials  and  processes  in  weapons  systems.  The  chapter 
provided  useful  examples  and  a  set  of  factors  that  are  used  to  mitigate  the  risk  of  using 
unreliable  parts  and  materials.  The  analysis  was  structured  around  five  management 
themes  and  attempted  to  pinpoint  either  practices  to  implement  in  a  program,  or  common 
pitfalls  that  PM  should  avoid.  Finally,  the  chapter  outlined  lessons  learned  from  the 
interview  process  and  the  background  data  gathered  as  part  of  this  research.  Chapter  V 
summarizes  this  topic,  draws  conclusions,  and  makes  recommendations. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

This  thesis  evaluated  the  current  process  for  managing  PM&P  Programs  on 
weapon  systems  and  identified  some  of  the  common  challenges,  pitfalls,  and  lessons 
learned.  The  key  issues  and  challenges  were  derived  from  interviews  with  individuals 
responsible  for  implementing  PM&P  Programs  within  the  Navy,  Air  Force,  and  Army 
and  with  their  contractors  on  weapon  system  programs. 

This  chapter  offers  conclusions  and  recommendations  for  managing  PM&P 
Programs.  It  also  identifies  the  correlation  between  highly  reliable  parts  and  materials,  as 
well  as  overall  reliability  and  critical  tactics  and  strategies  that  PMs  and  LMs  can  use  to 
maximize  the  inherent  reliabilities  of  system  components.  These  conclusions  and 
recommendations  are  based  upon  research  analysis  and  survey  responses.  This  chapter 
then  provides  brief  answers  to  the  primary  and  secondary  research  questions  and,  finally, 
makes  recommendations  for  further  study. 

B.  CONCLUSIONS 

Based  on  the  literature  review  and  an  analysis  of  interview  results,  the  researcher 
has  drawn  the  following  conclusions: 

1.  Idealistic  Reliability  Estimates  from  Contractors  Appear  to  be 
Systemic  in  Weapon  Programs 

One  of  the  greatest  challenges  facing  a  weapon  system  program  can  be 
establishing  the  reliability  of  the  equipment.  Program  and  Logistics  Managers  must  deal 
with  unrealistic  reliability  estimates  from  contractors,  who  are  under  pressure  to  win  and 
maintain  contracts.  The  impact  of  a  flawed  figure  of  merit  is  twofold.  Not  only  will  the 
weapon  system  be  considered  a  failure  during  testing,  but  the  development  of  a 
sustainment  package  based  on  a  flawed  reliability  factor  also  will  produce  sparing 
shortages,  test  equipment  shortages,  and  decreased  readiness. 
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2.  Highly  Reliable  Parts,  Materials,  and  Processes  are  a  Fundamental 
Part  of  Highly  Reliable  Weapon  Systems 

Simply  stated,  highly  reliable  parts,  materials,  and  processes  are  the  building 
blocks  of  a  highly  reliable  weapon  system.  While  the  system  design  approach  is  largely 
responsible  for  the  weapon  system’s  functional  performance,  reliable  parts  and  materials 
are  essential.  Therefore,  to  insure  that  the  reliability  performance  is  being  met,  one  has  to 
consider  and  evaluate  part  and  material  requirements,  their  implementation  and 
verification,  and  the  requirements  translation  to  the  lowest  part,  material  and  process 
applied  in  the  system. 

3.  Successful  Communication  is  a  Vital  Part  of  Optimizing  the  Inherent 
Reliability  of  a  Weapon  System 

Communication  must  be  emphasized  at  all  levels.  Communication  is  a  tool  that 
can  assist  the  program  office  in  avoiding  design  failures.  All  team  members  must  be 
aware  of  the  costs  of  disorganization  and  poor  communication.  A  weapon  system 
program  must  establish  an  atmosphere  of  open  dialogue  and  require  personnel  to 
participate  in  all  assigned  IPT  meetings. 

4.  A  PM&P  Program  Plan  is  a  Critical  Element  in  the  Design  of  a 
Highly  Reliable  Weapon  System 

There  must  be  a  clear  determination  of  the  system  perfonnance  requirements, 
including  reliability.  The  system  requirements  will  get  allocated  to  the  lowest  levels  in 
the  design.  The  impacts  of  not  establishing  a  proper  requirement  will  reverberate  down 
to  the  part  and  material  level.  A  PM&P  Program  Plan  is  critical  because  it  will  help  a 
program  know  and  understand  part  and  material  requirements. 

5.  Part  and  Material  Testing  is  an  Essential  Part  of  Weapon  System 
Development 

The  ability  to  design  a  reliable  system  requires  early  and  continuous  testing  from 
the  system  level  down  to  the  parts  and  materials  that  make  up  the  system.  This  testing  is 
necessary  in  order  to  verify  and  monitor  successful  achievement  of  all  the  performance 


58 


requirements,  including  reliability.  Programs  often  fail  to  perform  part  and  material 
testing  due  to  inadequate  upfront  reliability  testing  and  a  lack  of  funding. 

6.  Continuous  Monitoring  of  Suppliers  is  Crucial  to  High  Reliability 

Programs  need  to  monitor  suppliers  for  changes  to  parts,  materials,  and  processes 
that  would  affect  performance.  Attention  to  the  underlying  manufacturing  design  and 
manufacturing  processes  of  the  contractor,  their  subcontractors,  and  vendors  is  needed  to 
ensure  part  and  material  reliability 

C.  RECOMMENDATIONS 

The  following  recommendations  can  be  used  to  maximize  the  inherent  reliability 
performance  of  weapon  systems. 

1.  Establish  a  True  (Measured)  Reliability  Figure  of  Merit,  then 
Support  that  Number  and  not  Unrealistic  Contractor  Estimates 

During  initial  acquisition,  contractor  engineers  provide  reliability  estimates  that 
are  critical  in  establishing  sparing  levels  and  technical  support  criteria.  Experience  has 
shown  that  contractors  typically  use  optimistic  reliability  predictions.  [Ref.  17]  Weapon 
systems  must  dedicate  the  necessary  resources  to  verify  the  true  reliability  and  then 
support  that  number.  The  failure  of  a  weapon  system  to  achieve  its  projected  reliability 
levels  will  have  a  severe  impact  on  the  operation  and  support  (O&S)  costs  and 
operational  availability  (Ao). 

2.  “Design  in”  Reliability  Using  Highly  Reliable  Parts,  Materials,  and 
Processes 

The  use  of  highly  reliable  parts,  materials,  and  processes  to  “design  in”  inherent 
reliability  can  be  facilitated  with  the  following  two  techniques.  First,  weapon  system 
programs  should  establish  a  Part  and  Material  Review  Board  (PMRB).  The  PMRB 
would  be  responsible  for  implementing  effective  parts  and  materials  management  and  for 
promoting  commonality  of  parts  and  processes  across  product  lines.  The  PMRB  can 
enhance  the  implementation  of  concurrent  engineering,  by  including  representatives  from 
seven  areas:  1)  design  engineering;  2)  procurement;  3)  manufacturing;  4)  quality;  5) 
subcontractors  and  suppliers;  6)  customer;  and  7)  logistics. 
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Weapon  system  programs  should  also  require  their  contractors  to  establish  a 
FRACAS  that  requires  root-cause  corrective  action  at  the  part  and  material  level.  A 
FRACAS  is  an  important  tool  in  the  solution  of  complex  reliability  problems.  A  well- 
defined  FRACAS  will  catch  part  and  material  problems  early  in  the  life  cycle,  before 
they  cause  high-profile  failures. 

3.  Weapon  System  Programs  Must  Promote  Logistics  Involvement 

Most  weapon  system  programs  already  use  IPTs  as  a  management  technique. 
They  need  to  take  the  next  step  and  involve  the  logistics  manager  early  in  the  system  life 
cycle.  Most  weapon  system  programs  do  not  have  the  Logistics  Manager  involved  with 
the  PM&P  Program.  However,  weapon  system  programs  should  make  sure  that  a 
logistician  is  a  permanent  member  on  any  part-  and  material-related  IPT. 

4.  Know  and  Understand  Part  and  Material  Application  Requirements 

As  early  as  possible  in  the  system  life  cycle,  require  all  contractors  and 
subcontractors  to  develop  a  comprehensive  PM&P  Plan  based  on  MIL-HDBK-512  or 
American  Institute  of  Aeronautics  and  Astronautics  (AIAA)  industry  standard  document 
AIAA  R-100.  This  PM&P  Plan  should:  1)  explicitly  define  a  PM&P  Program  that  will 
ensure  the  use  of  parts  that  meet  contractual  requirements;  2)  reduce  proliferation  of  parts 
through  standardization;  and  3)  enhance  equipment  reliability  and  supportability. 

5.  Require  and  Support  Part  and  Material  Testing 

Weapon  system  programs  must  recognize  that  they  should  perform  qualification 
testing  at  the  part  and  material  level.  Funds  should  be  allocated  for  independent  testing 
of  piece  parts  and  materials.  The  component  engineers  must  know  that  there  are  no 
barriers  to  testing  of  parts  where  they  feel  it  is  required.  The  PM&P  Program  IPT  should 
establish  design  review  criteria  and  make  them  a  standard  part  of  the  design  review 
process. 

Weapon  system  programs  also  must  plan  for  PM&P  Program  activities.  Funding 
is  often  not  allocated  to  the  PM&P  Program  or  is  later  withdrawn  because  of  competing 
priorities.  A  long-term  approach  will  help  ensure  that  the  PM&P  Program  has  adequate 
funding  and  support. 
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6.  Continuously  Monitor  Suppliers 

A  weapon  system  program  must  ensure  that  contract  requirements  flow  down  to 
subcontractors,  suppliers,  and  distributors.  PM&P  Program  personnel  must  participate  in 
the  technical  evaluation  of  suppliers  and  in  the  review  and  approval  of  suppliers’ 
manufacturing  processes  and  parts  changes.  DoD  should  re-evaluate  the  policy  of  strict 
adherence  to  performance-based  requirements  and  require  contractors  to  participate  in 
programs  like  GIDEP.  Since  GIDEP's  inception,  participants  have  reported  more  than 
$1  BILLION  in  prevention  of  unplanned  expenditures.  This  means  that,  without 
GIDEP,  participants  could  have  potentially  realized  additional  expenses  exceeding  $1 
billion.  [Ref.  30] 

D.  ANSWERS  TO  RESEARCH  QUESTIONS 

Primary  Research  Question 

How  is  the  selection  of  highly  reliable  parts  and  materials  an  essential  part  of  the 
overall  reliability  development  process,  and  what  critical  tactics  and  strategies  can 
Program  and  Logistics  Managers  use  to  maximize  the  inherent  reliabilities  of  system 
components? 

Achieving  reliable,  available  and  maintainable  systems  requires  a  disciplined 
systems  engineering  approach  that  starts  early  on  in  a  program.  Maximizing  system 
reliability  is  something  that  can  be  achieved  with  the  help  of  highly  reliable  parts, 
materials,  and  processes.  Reliability  cannot  be  tested  in  later,  but  must  be  part  of  the 
design  process  while  it  is  still  cost  effective.  A  microcircuit  the  size  of  a  penny  can  end 
up  causing  an  entire  multimillion-dollar  weapon  system  to  fail.  The  following  are 
recommended  steps  a  Program  Manager  should  consider  taking  in  an  effort  to  optimize 
weapon  system  inherent  reliability: 

Provide  adequate  funding  for  a  PM&P  Program  that  includes  testing  at  the  part 
and  material  level. 

Develop  a  comprehensive  PM&P  Program  as  early  as  possible  in  the  system  life 

cycle. 
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Ensure  that  adequate  parts,  materials,  and  process  requirements  are  placed  in 
contracts. 

Involve  PM&P  experts  early  in  the  process. 

Implement  a  FRACAS  down  to  the  part  and  material  level. 

Monitor  and  emphasize  communication  with  suppliers. 

Utilize  IPTs,  including  a  PM&P  IPT. 

The  following  are  recommended  steps  a  Logistics  Manager  should  consider 
taking  in  an  effort  to  optimize  weapon  system  inherent  reliability: 

Make  sure  that  the  design  IPTs  understand  the  logistics  impacts  of  the  early 

design. 

Make  sure  that  the  PM  understands  the  logistics  impacts  and  importance  of 
reliability. 

Provide  feedback  at  the  part  and  material  level,  from  the  field  and  depots. 

Provide  education  for  your  people.  Most  logisticians  lack  the  technical  skills 
needed  to  impact  the  early  design. 


Subsidiary  Research  Questions 

The  following  subsidiary  questions  focused  the  author’s  efforts  in  answering  in 
the  primary  research  question. 

a.  What  technology,  tools,  and  techniques  are  available  to  ensure  the  proper 
selection  of  reliable  parts  and  materials? 

Many  tools  and  techniques  can  be  used  to  insure  the  proper  selection  of  reliable 
parts  and  materials.  Listed  below  are  the  most  frequently  utilized  design  tools  to  ensure 
that  reliability  is  “built  in”  early  in  the  design  of  a  weapons  system: 

Reliability  Prediction  Analysis 

Circuit  Tolerance  Analysis 
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Sneak  Circuit  Analysis 

Physics  of  Failure  Tools 

Part  Derating 

Government  Industry  Data  Exchange  Program 

b.  How  has  acquisition  reform  changed  the  way  a  PM&P  Program  is  managed? 

Acquisition  reform  has  significantly  increased  the  risks  to  PM&P  Programs.  Prior 
to  acquisition  refonn,  a  PM&P  Program  was  strictly  controlled  by  the  Government 
through  contractually  required  military  specifications  and  standards.  After  acquisition 
reform,  military  specifications  and  standards  could  no  longer  be  used.  There  was  no 
longer  one  uniform  process;  there  was  the  loss  of  lessons  learned  intrinsic  to  the  military 
specifications  and  standards,  and  all  requirements  had  to  be  specified  in  performance- 
based  terminology.  This  has  placed  a  premium  on  the  ability  to  communicate. 
Concurrent  engineering  and  the  use  of  IPTs  are  powerful  management  techniques  that 
can  reduce  some  of  the  risks  that  were  created. 

c.  To  what  extent  does  commercial  industry  differ  in  its  approach  to  part  and 
material  selection,  and  can  the  military  leverage  these  best  practices  to  improve 
performance  in  military  systems? 

The  DoD  already  leverages  off  commercial  practices,  where  practical,  in  its  part 
and  material  selection.  It  also  is  apparent  that  commercial  industry  has  not  stepped  up 
and  developed  “best  practices”  or  initiatives  to  replace  part  and  material  military 
standards  and  specifications.  Where  part  and  material  requirements  and  testing  are 
concerned,  commercial  industry  leverages  off  the  DoD.  Most  of  the  differences  between 
the  military  and  commercial  industry  can  be  attributed  to  a  different  business 
environment.  There  are  also  basic  design  differences  between  parts  and  materials  used 
for  military  vs.  commercial  applications.  Most  military  designs  are  used  in  very  harsh 
environments,  whereas  most  commercial  components  are  designed  for  a  protected 
environment.  Commercial  parts  and  materials  many  times  are  not  subjected  to  shock, 
vibration,  pressure  or  moisture  and  have  an  application  life  span  of  less  than  five  years. 
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The  military  environment  is  much  more  severe  and  requires  parts  and  materials  with  an 
application  life  span  of  ten  to  20  years.  [Ref.  31] 

d.  How  can  Program  and  Logistics  Managers  mitigate  part  and  material 

risks? 

The  most  important  thing  a  PM  can  do  to  mitigate  part  and  material  risks  is  to 
make  maximum  use  of  IPTs.  IPTs  will  increase  the  flow  of  communication  and  promote 
the  involvement  of  parts  and  materials  engineers  at  the  earlier  stages  of  design.  The  most 
important  thing  an  LM  can  do  to  mitigate  part  and  material  risks  is  to  get  involved  early 
in  the  IPT  process.  The  LMs  also  need  to  develop  the  technical  expertise  to  influence  the 
design  of  the  product. 

E.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

The  following  are  recommended  topics  for  additional  research: 

1 .  Evaluate  the  kind  of  data  currently  collected  in  the  field  and  detennine  how  to 
best  provide  feedback  to  a  PM&P  Program  so  there  can  be  improvement. 

2.  Analyze  the  best  methods  and  approaches  for  incentivizing  reliability,  not 
only  in  contracts,  but  also  in  project  offices. 

3.  Perform  a  more  complete  study  on  how  commercial  industry  manages  parts, 
materials,  and  processes.  Assess  if  anything  can  be  adapted  for  weapon 
system  development. 

F.  THESIS  SUMMARY 

Managing  reliability  perfonnance  on  a  weapon  system  demands  the 
implementation  of  effective  management  strategies  that  balance  cost,  schedule  and 
performance  against  reliability  risks  over  the  course  of  a  weapon  system’s  development 
and  fielding.  Although  reliability  is  composed  of  many  parts,  one  of  the  keys  resides  in 
early  identification  of  upfront  cost-effective  opportunities  for  improving  reliability 
performance,  as  well  as  mitigation  of  associated  risks  during  design.  Predictable  and 
reliable  performance  in  the  field  is  the  desired  end  state. 

Highly  reliable  parts,  materials,  and  processes  are  the  building  blocks  of  a  highly 
reliable  weapon  system.  While  the  system  design  approach  is  largely  responsible  for  the 
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weapon  system’s  functional  performance,  the  PM&P  Program  approach  is  responsible  for 
a  large  part  of  the  weapon  system’s  reliability  performance.  Therefore,  to  insure  that  the 
reliability  perfonnance  is  being  met,  one  has  to  consider  and  evaluate  the  PM&P  Program 
requirements,  its  implementation  and  verification,  and  the  requirements  translation  to  the 
lowest  part,  material  and  process  applied  in  the  system.  The  goal  of  a  PM&P  Program  is 
the  selection  of  parts,  materials,  and  processes  that  can  withstand  the  manufacturing 
environment  and  reliably  perform  the  needed  function  for  the  design  life  of  the  product  in 
the  intended  environment. 
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